Workflow Designer | The Operation has Timed out

Question

When I try connecting the CM from Visio Workflow Designer by double clicking the any WF activity, the designer pops up a dialog to enter credentials and after submitting the credentials, I am getting the exception as below after a minute or so.

The operation has timed out

System.Web.Services()

I checked logs, it looked a timeout issue to me so I increased timeout values (to 03.00.00 ) for "CoreService_netTcpBinding" binding in "web.config" at path "{TridionInstallDirectory}\Web" and in "TcmServiceHost.exe.config" at path "{TridionInstallDirectory}\bin", but getting the same error as below:

The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '03:00:00'. The write operation failed, see inner exception. The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '03:00:00'. An existing connection was forcibly closed by the remote host Component: Tridion.ContentManager.CoreService Errorcode: 0 User: NT AUTHORITY\NETWORK SERVICE StackTrace Information Details: at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) at System.ServiceModel.Channels.SocketConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) at System.ServiceModel.Channels.BufferedConnection.WriteNow(Byte[] buffer, Int32 offset, Int32 size, TimeSpan timeout, BufferManager bufferManager) at System.ServiceModel.Channels.BufferedConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) at System.ServiceModel.Channels.ConnectionStream.Write(Byte[] buffer, Int32 offset, Int32 count) at System.Net.Security.NegotiateStream.StartWriting(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.NegotiateStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.NegotiateStream.ProcessWrite(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.NegotiateStream.Write(Byte[] buffer, Int32 offset, Int32 count) at System.ServiceModel.Channels.StreamConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) at System.ServiceModel.Channels.StreamConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout) at System.ServiceModel.Channels.StreamConnection.Write(Byte[] buffer, Int32 offset, Int32 size, Boolean immediate, TimeSpan timeout, BufferManager bufferManager) at System.ServiceModel.Channels.FramingDuplexSessionChannel.OnSendCore(Message message, TimeSpan timeout) at System.ServiceModel.Channels.TransportDuplexSessionChannel.OnSend(Message message, TimeSpan timeout) at System.ServiceModel.Channels.OutputChannel.Send(Message message, TimeSpan timeout) at System.ServiceModel.Channels.TransactionDuplexChannelGeneric`1.Send(Message message, TimeSpan timeout) at System.ServiceModel.Channels.RequestContextBase.Reply(Message message, TimeSpan timeout) at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.Reply(MessageRpc& rpc)

AND

Thread was being aborted.

Component: Tridion.ContentManager.Workflow.DesignerService Errorcode: 0 User: NT AUTHORITY\NETWORK SERVICE

StackTrace Information Details: at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message) at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type) at Tridion.ContentManager.CoreService.Client.ISessionAwareCoreService.GetSystemWideList(SystemWideListFilterData filter) at Tridion.ContentManager.Workflow.DesignerService.WFListener.GetSystemWideList[T,FT](ListBaseColumns baseColumns, FT other) at Tridion.ContentManager.Workflow.DesignerService.WFListener.GetWorkflowData()

Am I missing increasing the timeout value somewhere or it's unrelated to increasing timeout?

UPDATE

While connecting, it looks like Tridion iterates through and performs several different Tridion.ContentManager.Workflow.DesignerService.WFListener.GetSystemWideList() methods. These methods in turn make a database call and read data back as a result of the call. the final iteration of the GetSystemWideList() method makes the following database call:

TCM_WORKFLOW_GETLIST_PROCESS_DEFINITIONS

While the database call itself take less than 100ms to complete, it looks like the resulting data read takes a whopping 2+ minutes to complete in Sandbox.

I tried browsing Workflow Process Definitions in Tridion Browser. And it take around 2 mins to respond but present the results in the end without timing out.

Answer 1

So to fix the issue we did the following:

1. Turned the Tracing off on CMS Server.
2. Our DB was of 2013 version and not in sync with Tridion Version 2013SP1.

I don't think it was because of the point 2, because it was working fine from long back with the same setup. So I guess, it was the Tracing. I also noticed, we had huge logs(in GBs) on CMS server and purged those later.

Answer 2

assuming:

• you can connect to other environments (this reduces the likelihood of it being Visio or Workflow Plugin related)

• you have all logs on DEBUG and see nothing other than what you've posted (including the windows event logs)

• your user still has the correct permissions and you can perform the usual functions in Tridion (such as triggering a GetList for a Publication list etc.)

• everything looks good as far as IIS, Services, COM component

• you can see that there was no Windows/JAVA/other updates since Visio / workflow last worked

  .

• I think it's unnecessary but it's a step that Support would ask you to take - I'd uninstall and reinstall the plugin (and confirm all is well connecting to another environment still before moving on)

Then, I'd:

• run sp_updatestats (this should be running on a schedule anyway - even on a sandbox machine - I'd ask DevOps to rectify this if it isn't the case)
• I'd be reluctant to start cleaning up in the hope if fixes a seemingly unrelated issue (but nice to flag purge scripts and other DB Maintenance as general good practice later)
• at this point, I'd be starting a Support Ticket
• when/if sp_updatestats doesn't work I would work with DevOps and put a TRACE on the SQL DB - watch what SQL queries are executed (if any - given it seems some web service isn't responding - it may still be making valid requests)
• if SQL TRACE shows nothing it may help to dig deeper into what's going on behind the scenes (using a tool such as DynaTrace) with - catching the error at that deep level may surface some error report you're not otherwise seeing