Taxonomy Broker cache issue in Tridion 2009

Question

This is a Tridion 2009-SP1 Java environment with an Oracle Broker. Both the deployer and the website are running on the same server. We have the CD_2009.1.71453 roll up hotfix installed as well

We use some Keywords with metadata to store and retrieve information dynamically.

When we turn the CacheChannelService on with all the proper cacheBindings for the different dynamic content types it works for all but the Taxonomies.

What we can see happening is that when the cache is empty (after a restart or manual clearing) that the keywords are properly added to the cache after a publish of the containing category. However, when we update a keyword or add a new one and republish the category this does not get picked up in the website. It even causes an error when it can't find the new keyword in the cache.

It seems that somehow the mechanism where;

1. It checks the cache
2. Not in the cache (or updated version)
3. Retrieve item from the Broker

Does not work properly.

Either we are missing something really obvious or it is a bug. I have looked through all the other posts about caching, but haven't found the solution yet. Tridion support has been contacted, but so far they haven't given the golden tip either.

Simple version of Broker conf.

cd_broker_conf (same on uploader and website):

<Bindings>
  <Binding Name="LinkInfo" Class="com.tridion.broker.linking.SQLLinkInfoHome" />
  <Binding Name="ComponentMeta" 
           Class="com.tridion.broker.components.meta.OracleComponentMetaHome" />
  <Binding Name="Taxonomy" Class="com.tridion.broker.taxonomies.SQLTaxonomyHome" />
</Bindings>
<ObjectCache Enabled="true">
  <Policy Type="LRU" Class="com.tridion.cache.LRUPolicy">
    <Param Name="MemSize" Value="128mb" />
   </Policy>
  <Features>
    <Feature Type="DependencyTracker" Class="com.tridion.cache.DependencyTracker" />
  </Features>
  <CacheBindings>
    <CacheBinding Name="LinkInfo" Class="com.tridion.broker.linking.CachedLinkInfoHome" />
    <CacheBinding Name="ComponentMeta"
                  Class="com.tridion.broker.components.meta.CachedComponentMetaHome" />
    <CacheBinding Name="Taxonomy" Class="com.tridion.broker.taxonomies.CachedTaxonomyHome" />
  </CacheBindings>
  <RemoteSynchronization Host="127.0.0.1" Port="1099" Queuesize="10000"
                         ServiceMonitorInterval="10000" />
</ObjectCache>

Added the part of the cd_broker.log where you can see it looking for a new keyword in the cache:

[Debug 27-03-2013 09:35:41] Retrieving Keyword with KeywordURI: tcm:11-12436-1024
[Debug 27-03-2013 09:35:41] Retrieving Keyword List from cache
[Debug 27-03-2013 09:35:41] Retrieving Keyword with KeywordURI: tcm:11-12436-1024
[Debug 27-03-2013 09:35:41] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Retrieving Keyword with KeywordURI: tcm:11-12799-1024
[Debug 27-03-2013 09:36:50] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Retrieving Keyword with KeywordURI: tcm:11-12801-1024
[Debug 27-03-2013 09:36:50] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Retrieving Keyword with KeywordURI: tcm:11-12801-1024
[Debug 27-03-2013 09:36:50] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Retrieving Keyword with KeywordURI: tcm:11-12800-1024
[Debug 27-03-2013 09:36:50] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Retrieving Keyword with KeywordURI: tcm:11-12444-1024
[Debug 27-03-2013 09:36:50] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Retrieving Keyword with KeywordURI: tcm:11-12451-1024
[Debug 27-03-2013 09:36:50] Retrieving Keyword List from cache
[Debug 27-03-2013 09:36:50] Cache miss, Keyword list not in cache, retrieving and storing in cache
[Debug 27-03-2013 09:36:50] Retrieving a keyword: SELECT TF.FACET_ID, TF.PUBLICATION_ID, TF.TAXONOMY_ID, TF.FACET_NAME, TF.FACET_PARENT, TF.FACET_ISABSTRACT, TF.FACET_ISNAVIGABLE, TF.FACET_HASCHILDREN,  TF.FACET_KEY, TF.FACET_LEFT, TF.FACET_RIGHT, TF.FACET_DEPTH, TF.FACET_DESCRIPTION, TF.FACET_ITEMTYPE,(  SELECT COUNT(DISTINCT ITC.ITEM_REFERENCE_ID) AS CNT FROM ITEM_CATEGORIES_AND_KEYWORDS ITC WHERE ITC.KEYWORD_ID=TF.FACET_ID ) AS RELATED_ITEMS  FROM TAXFACETS TF LEFT JOIN ITEM_CATEGORIES_AND_KEYWORDS ITC ON ITC.KEYWORD_ID=TF.FACET_ID WHERE TF.PUBLICATION_ID = ? AND TF.FACET_ID = ? AND TF.FACET_ITEMTYPE = ? GROUP BY TF.FACET_ID, TF.PUBLICATION_ID, TF.TAXONOMY_ID, TF.FACET_NAME, TF.FACET_PARENT, TF.FACET_ISABSTRACT, TF.FACET_ISNAVIGABLE, TF.FACET_HASCHILDREN, TF.FACET_KEY, TF.FACET_LEFT, TF.FACET_RIGHT, TF.FACET_DEPTH, TF.FACET_DESCRIPTION, TF.FACET_ITEMTYPE
[Debug 27-03-2013 09:36:50] No keywords found for the given Keyword URI: tcm:11-12451-1024

Answer 1

Looking closely at your problem, it seems that the findTaxonomyKeyword method from the CachedTaxonomyHome class caches your result but when the whole taxonomy is reDeployed, the cache invalidation does not happen. To fix this you have two options:

1. You contact Tridion Customer Support, explain the problem, explain why this would be a ShowStopper for your project and they might consult with R&D to see if a hotfix would be required;

2. You implement/configure your own version of CachedTaxonomyHome in which you add a dependency to make sure the cache is invalidated when needed. Below you will find an implementation example for this:

public class FlushingCachedTaxonomyHome extends CachedTaxonomyHome {
    private Logger log = Broker.getInstance().getLogger();
    private TaxonomyHome taxonomyHome;
    private Cache taxonomyCache = CacheFactory.getInstance().getCache(DistributorCacheConstants.REGION_TAXONOMY);

    public FlushingCachedTaxonomyHome(Object brokerHome) {
        super(brokerHome);
        this.taxonomyHome = (TaxonomyHome) brokerHome;
    }

    public List findTaxonomyKeyword(int publicationId, int keywordId, int keywordItemType) throws StorageException {
        log.debug("Retrieving Keyword List from cache");
        Serializable key = KeyGenerator.createKey(publicationId, keywordId, keywordItemType);
        List foundKeywords = (List) this.taxonomyCache.get(key);
        if (foundKeywords == null) {
            log.debug("Cache miss, Keyword list not in cache, retrieving and storing in cache");
            foundKeywords = this.taxonomyHome.findTaxonomyKeyword(publicationId, keywordId, keywordItemType);
            this.taxonomyCache.put(key, foundKeywords);
            if (foundKeywords != null && !foundKeywords.isEmpty()) {
                final String taxonomyURI = ((Keyword) foundKeywords.get(0)).getTaxonomyURI();
                try {
                    final TCMURI taxonomy =  new TCMURI(taxonomyURI);
                    final int taxonomyId = taxonomy.getItemId();
                    // add a dependency for our added taxonomy keywords to the actual taxonomy to which they belong
                    this.taxonomyCache.addDependency(key,
                              DistributorCacheConstants.REGION_TAXONOMY, KeyGenerator.createKey(publicationId, taxonomyId));
                } catch (ParseException pe) {
                    //TODO: handle this
                }
            }
        }
        return foundKeywords;
    }
}

Keep in mind that this is only an example but it should get the job done. Also, do not forget to add it to the CacheBinding.

Answer 2

I'd like to comment a statement in the thread of the answer above:

"... but mainly for the deployer. Why the deployer? Because the one that is actually sending the invalidation messages is the Deployer? "

Looking in the log files this statement does not seem to be true... It must be implemented in the web app. But better, as suggested in both: uploader/web app.

And one more suggestion/question: Shouldn't:

this.taxonomyCache.put(key, foundKeywords);
//be after the check for null:
if (foundKeywords != null && !foundKeywords.isEmpty()) {
?

It could become a showstopper if you publish (and visit) an item that reads a Keyword that's not published yet...