Overriding Publishing Queue

Question

We have a requirement to override publishing queue of Tridion (2013 SP1 and SDL Web 8) where we need to take care of below scenario.

Scenario :

1. If there are multiple items in publishing queue, we should be able to publish any item before other items present in publishing queue. [It should be achievable by setting publishing priority]

2. If publishing queue already contains item like Bundle/Structure Group containing multiple items, we should be able to publish item before the structure group gets published.

Probable Solution:

1. Have a separate publishing channel for such items. http://blog.building-blocks.com/technical-tips/filtering-prioritization-of-the-tridion-publish-queue
2. Using event system abort items present in publishing queue and publish current item and re-publish previous items.

Answer 1

For #1 publishing priority is the way to go. Maybe changing the default priority to "low" will give you enough of an override to push things through quickly. Using publishing priorities in conjunction with the information I share in the blog post on keeping threads available.

Item #2 might need event system, but you should be careful of performance impacts. Querying a large publish queue to find if an item is already in the queue might be slow.

My bigger question is why do you need this, and can its need be removed in another way? For example:

1. Are templates slow to render/publish - check for template optimizations etc
2. Is the queue regularly very long - consider scaling the publisher and reviewing why you need to publish such high volumes (maybe architectural mistake?)

I'd suggest reviewing the root cause before applying these Band-Aids.

Answer 2

What you are requiring is mainly to use publish queue in a way its not meant to be used.

For #1 you want to publish items before all others. That is the exact purpose of "high" priority. Problem is when editors abuse this priority so you have thousand of items in queue on high.

Using event system to remove/resend transactions can be very time and resource consuming so I wouldn't suggest it, unless really necessary. What I suggest is to create detail analysis of publishing:

1. Who and what should publish
2. Should all users have same publishing priorities (you can create gui extension to dissable high priority for certain users).
3. Should more out-scaled publishers be added

To manipulate publishing queue you can also create core service script which can be ran at request (add/remove/update transactions). It can be ran at off hours so to have less impact on your CM server.

Answer 3

It seems you want to prevent big Publish Transactions to be queued with High priority, to ensure that urgent, small ones can get priority (user manually sets High priority) and will be processed quickly.

You could create a publish event handler which automatically sets the priority for "big" publish actions to Medium or Low (even if the user set it to High).

In it's simplest form, it could do that for all but Pages. If you want it to be smarter (publishing a small SG or Bundle should maybe be allowed with High prio?), you could use ResolveEngine.Resolve to determine how many items will get rendered. However, note that resolving can be quite expensive and a publish event happens synchronously when the user performs a publish action (i.e. the user has to await your event handler).

Indeed, if you want to guaruantee that small, high prio Publish Transactions are not blocked by lower prio big ones, you would also have to reserved publisher capacity by letting one or more publishers only pick up high prio ones.