Please note: updated answer!
The JMS-based cache invalidation of the .NET version of DD4T in a pre-release, so you have to do a bit of work.
This is what you should do:
Add a NuGet reference to DD4T.Caching.ApacheMQ (set it to prerelease mode since there is no official release yet)
Add the following line to the Application_Start method, before you call builder.UseDD4T()
When you have set the DD4T references using builder.UseDD4T(), you need to start the message provider and subscribe your CacheAgent to it, like this:
var cacheAgent = diContainer.Resolve<ICacheAgent>();
if (cacheAgent is DefaultCacheAgent)
The message provider will now listen to the configured address (using the DD4T.JMS.xxx settings that you have already) for invalidation messages which are broadcast by the deployer.
But there is also a change you need to do on the deployer end. The DD4T framework contains a little-used customization of the Tridion CacheChannel logic. To implement, do this:
Download this source code and build it using some Java IDE: https://github.com/dd4t/dd4t-cachechannel
Create a jar file from it
Copy the jar file to the lib folder of the deployer
In the cd_storage_conf.xml of the deployer, locate the Connector element inside the RemoteSynchronization section.
Change it so it uses the bespoke connector, like this
<Connector Class="com.tridion.cache.TextJMSCacheChannelConnector" Topic="Tridion" Strategy="AsyncJMS11">
It should now broadcast text messages instead of binary messages. This is needed because the binary messages cannot be consumed through .NET.
As I said, it won't be easy. If you manage it, please blog. It will make it easier for the rest :)