Faceted search using Tridion Categories and Keywords

Question

I need to design a faceted search against a number of products. The content is tagged up with metadata using Categories and Keywords. Normally I'd use something like Elasticsearch or SOLR to deal with the faceting based on this metadata but in this case it's not an option that's available to me.

Speaking to various people, I'm hearing a general theme that the API is sub-optimal for doing this sort of work and you'd have to construct the facets yourself from the taxonomy using many API calls. I appreciate the genericity of this question but has anyone successfully implemented this? What are the pitfalls?

Answer 1

Even though this question seems to be based a lot on "hearsay", I'll give you my best shot.

Can Tridion Categories and Keywords be used to build a taxonomy-driven website? Absolutely YES.

I know of two things you need to pay attention to:

• Dimensions on your taxonomy
• Cache

Dimensions

A taxonomy can have many types of relationships defined. You can have the basic "tree-like" hierarchy, you have the "related keywords" concepts, and you have potentially additional keyword fields in keyword metadata. When a category is published Tridion will try to create an OWL representation of this. I've heard of one case where about 40,000 keywords were expanded to nearly one million representations in OWL. Publishing this category could take up to 12 hours (!) mostly on the deployer side. Clearly a situation to try avoiding. So, manage your dimensions.

Cache

Tridion will do its best to load the taxonomy only once and keep it in memory as much as possible, as long as cache is enabled. Do NOT even try to do this without cache enabled. Yeah, I hear you saying "of course", but you'd be surprised how many Tridion sites run without cache (it's great for consulting though, you can fix a seemingly complex performance issue within 2 hours).

There's quite a few sites out there using taxonomy, and its performance improved significantly in 2011 SP1 and later.

It has to be clear that Tridion Taxonomies are not meant to replace real taxonomy-management tools that sell by prices that dwarf Tridion license costs. For those, you have specialized Product Information Management tools that can deal with Million+ records and associations and still deliver results in the low millisecond range (tools like SDL's Fredhopper).

Answer 2

I actually think in this scenario the answer is 'Build a POC', here's why:

• There's many ways to skin a cat with the CD api
• You've already got your content tagged with categories and keywords
• Publish it, then I reckon about 2 hours to knock together some code to build the POC
• You've then the ability to play with the cache and check performance

When we've clients struggling with time and budgets, and the tasks is SDL Tridion related and something we don't know the details of, we get together out of hours to work on it.. client's will value your knowledge and commitment to stay on top of the product and your bosses (i hope) will admire your work ethic, sharing back with the community will also gain you even more love :)

Answer 3

A classic discussion of this problem can be found at the Wiki

The pertinent quotation is this: "Architects don't have to code, as long as they have ways to get concrete feedback on their ideas."

I'd suggest that if speaking to people and extracting general themes isn't getting you concrete feedback on whether there is a Tridion API suited to the task you have in mind, your approach should be as follows:

1. Write some code to test your ideas.
2. If necessary, ask questions here to assist with point 1, and maybe to take the thinking further.

tridion.stackexchange.com is probably not a great forum for making your issue concrete. That part you need to do yourself. I know you acknowledged the generalness of your question, but without you first fleshing out the bones of what you want to achieve and how you have tried and failed, the question defies answering.

Answer 4

I've come to the conclusion after some research that the API is next to useless for building high performance, scalable faceted search and will burn hours and hours of time

• re-inventing a wheel
• getting performance up to scratch.

I have done a small POC of the front end and there's no doubt ES is a much better idea so need to convince the customer to do this.

I am intending to use the SI4T baseline and then modify for ES - however, I do have one further question - is the way SI4T is implemented a 'best practice'? there is some debate on Deployer extensions versus Storage extensions although I'm not sure this applies for 2011 SP1?