Why is deployment feedback not reflected in the publish queue?

Question

When publishing, we see items go to Waiting for deployment and then remain there. In the content delivery logging, we can see that deployment is complete, and in the transport log on the CM we can see that the deployment appears to be successful.

On a working system, we can see that in the transport log, there are several polling attempts before finally a success status is reached, and the state file is remotely deleted. On our problematic system we see that success is reached on the first polling attempt. (Both behaviours could be normal, perhaps depending on timing.)

The problem is that even though we can see success reported in the transport log, the item remains in the publish queue at Waiting for deployment. What is the cause likely to be?

Version: 2013 SP1 HR1

Update: The problematic system started out as a scale-out, but we've stripped it back down to a simpler scenario. So we have a single deployer, and we're conducting our tests with a single publisher running. The working systems have, as far as we can tell, the same version of everything installed. The transport runs over https when it leaves the CM, but the secure connection ends at a network device before it reaches the deployer application, so the deployer application sees plain-old HTTP coming in. As far as we can tell this is transparent to Tridion, although we're looking at confirming this. For the rest, as far as we know, the environments are very similar.

Update: As a test, we have configured the publisher from the working environment to use the database of the problematic environment. We can successfully publish this way. This seems to point clearly to a problem in CM rather than in CD.

Answer 1

I seen this issue where items are stuck in "Waiting for deployment" when in fact it's successful when there are many stale state files in the deployer's incoming folder. Cleaning the folder and restarting the service/site fixed it for me.

I also seen cases where the Broker DB version did not match the deployer, and caused this. For example, deployer is using 2013 jar files while broker db is still 2011.

Answer 2

I've seen this happen when using multiple Publishers & Deployers.

The publisher service polls the deployer to see if the publish completed successfully. If Publisher 1 sends the package over to Deployer 1, however Publisher 2 polls for the success and polls Deployer 2 you can see this.

Ultimately the cause is the publisher not being able to poll the deployer to determine if the publish transaction completed correctly. Using a monitoring tool like Fiddler on the Publishing server might help troubleshooting.

These links might help:

http://blog.building-blocks.com/technical-tips/scaling-the-sdl-tridion-deployer http://www.julianwraith.com/2012/12/multi-multi-sdl-tridion-deployers-solving-the-race-condition/

Answer 3

The issue seems to me that the Transport Service has stopped polling for a status and the last status it got from the deployer was that the item(s) were waiting for deployment.

This can happen in particular when doing a mass publishing action, as the deployer has a lot of work to do and probably picks up the package later than the transport has sent it.

I would increase the timeout in the cd_transport_conf.xml from the default 15 minutes to 30 minutes and for most customers that is usually sufficient.

Polling MaxAttempts="900" Timeout="30" Interval="5000"

Additionally, if you are using a protocol different from HTTP, it could be related to the senders timeout that is not sufficient. In that case, you can increase the Senders timeout to a higher value than the default (1 minute):

Senders SendTimeout="inmilliseconds"

hope this helps!

Answer 4

It turned out that the MSDTC on the publisher server could not communicate correctly with the MSDTC of the database cluster. We found messages in the Windows event log indicating this, and were able to follow up with further diagnostics using DTCPing. After reconfiguring the firewall, it works.

Answer 5

The fact that the transport log on the CM server indicates it received success feedback from the deployer rules out issues with the deployer itself afaict.

In essence everything is working normally until the transport service needs to update the status of the publish transaction with the deployment feedback.

Can you verify that the transport service user has sufficient rights/maps to a valid Tridion user?

It would also be useful to verify that the transport service itself is working normally with other deployers (maybe use a dummy local filesystem deploy for testing purposes?).