An answer to your question.
Question: Do we need to set it up from Web Application end as well?
Answer: Not mandatory, up to your business requirements.
If you using DD4T 2.0 Web application then good you can setup this ActiveMQ for caching invalidation with Deployer and Web application, You don't need to use both CIS caching and DD4T Web application caching for ActiveMQ caching invalidation.
Question: Why Deployer and Content Service Configuration with ActiveMQ jar is not Enough, why I need to change my Application for ActiveMQ?
Answer: If you use DD4T Web application with time-based caching, In this scenario, you can use deployer and CIS with ActiveMQ for caching invalidation, then you don't need to change your web application to integrate ActiveMQ.
Note:
If you decided to use DD4T web application time based caching then Content Editor should aware of this web application caching time in order to wait to see their changes on the web application, if you use DD4T web applications with ActiveMQ for caching invalidation integration then changes will be immediately visible on the web application.
Updated:
Question: if I use Deployer and CIS with ActiveMQ for Caching invalidation, then my Caching would be updated as soon as I publish, right?
Answer: yes, say for example web application you have setup 5mins caching then, web app cache will get clear after that 5mins caching expire, but CIS caching will get the cleared immediately.
Question: if Application uses DD4T Cache, and the cache is updated as soon as I publish, then what is the use of Web Application changes?
Answer: If your DD4T web application not used ActiveMQ integration, then No, it will wait to clear based on the time which you have configured.
I hope it helps to understand.