We have a DXA 2.0 web application with Web 8.5. We have multiple pages on this application as well as DCPs. The pages are published as a normal DXA workflow to the Broker DB. They are working as expected.
One of those pages has products being displayed based on metadata entered by the user in a form field. To accomplish this we setup the products to be pulled as DCPs that the web application queries for after the user enters the metadata value.
Both the regular pages and the DCPs work perfectly fine on Dev and QA environments. When we publish new changes, they reflect on the website immediately. Our Dev enrionment has no caching, but our QA encironment has both object cache and ehcache enabled. On staging we can see the DCP products and the regular pages both update after 15 minutes (the time of our ehcache). The issue is on Production, any update we make to the DCP content doesn't reflect on the website. The content service API shows the updates as soon as we publish, but the website doesn't. The ehcache expiry is set to an hour, but it has been a week now and the updates still don't show up. The regular pages work fine and reflect the content update after the ehcache expires.
We also tried rebooting the webapp to clear the ehcache but we found the DCP content still did not update to our latest changes.
What could be causing this issue where the queried DCPs don't show updated content on the website?