Possibility to clear the Publishing queue in Web 8 in case of bulk publishing by a specific user

Question

This is an open question. In our project we have multiple applications hosted from our SDL WEB 8 CMS. Currently, when any user does a bulk publishing, 20/30 items (page/components) at a time, the publishing gets real slow. All items remain in "Waiting for Publish" state for a long time. The queue has to be cleared to again enable successful publishing.

Is there a way to automatically clear the queue of items from a particular user when they push more than, say, 20/30 items for publish?

Please advice on a probable solution for this if any. TIA

Answer 1

Addressing a few things first.

1. If you see items stuck in "Waiting for Publish" it means that your publisher(s) don't have enough render capacity. Consider changing the number of render threads (based on actual CPU you have) to improve this.
2. No, you cannot out-of-the-box remove items from publishing queue simply because people are trying to publish a lot of things (they may have a very valid business reason for that)

Now, to answer your question of

Is there a way, to automatically clear the queue, of items from a particular user, when they push more that say 20/30 items for publish?

Customization necessary. You could easily write a Core-Service based tool that checks the queue regularly (every 60 seconds?) and checks if there's too many jobs pending. I still believe that deleting those transactions is counter-productive, but you could use this tool to change the priority of those transactions to "Low Priority".

If it is a specific user you have trouble with... maybe just talk to him? Or consider adapting his permissions to the Target Types.

Answer 2

An addition to Nuno's answer basically, since he is right on indicating that if the problem is too many items in the queue, you are basically looking at a publishing performance which is too low for your usecase.

But keep in mind, you might also want to inspect why the publishing actions themselves are so slow, and why so much is being added to the queue. Template Rendering performance might need to be looked at. Also Author education about the Content Process could be required (depending on if all those publish actions are indeed required or not).

Last there is a tool to purge the queue (see the documentation), but that is most likely NOT something you actually want to use in this case!

Answer 3

Deleting the items from the queue will only annoy your editors, causing them the republish the items anyway, but 20 or 30 items is not what I would call bulk. Tridion publishing will scale to huge numbers of items.

It really sounds like you do not have sufficient publishing capacity configured to meet the demands of your editors.

The default settings with 1 publisher machine would only process two items at the same time. It is easy to reconfigure this to publish 4, 8 or 16 items at a time to handle your publishing volumes.

Options:

1. Configure more publishing threads, default is 2 but you can probably up this to 2 * number of machine cores, or more just as long as you check the CPU loading is ok when publishing.

2. Configure more publisher machines.

3. Configure separate publishers for stage and live, use the publisher on the CMS only for stage and add another for live.

4. Configure separate publishers for publishing groups of publications.

5. Investigate publishing performance and fix slow template code.

6. Use workflow to publish content and defer non essential publishing to later in the day.

7. Use event system to defer publishing to later in the day.

8. Publishing will add extra load to your cm database, check that your cm database is not holding up publishing.

But at the end of the day, user education is the cheapest solution, remind them that they are sharing the queue.