Take the 2-minute tour ×
Tridion Stack Exchange is a question and answer site for Tridion developers and administrators. It's 100% free, no registration required.

I am working with legacy implementation in R5.3 SP1 and there are more than 100 websites most of them have dynamic content serve through the Broker API. We have a 3 Server Load Balancing set up - say Server 1 , Server 2 and Server 3. Here Server 1 is primary and publishing happens only on it and Server 2 and 3 are replicated from it using DFS. My concern is that almost daily Server 2 and 3 stops responding to web request while Server 1 is responding quickly.

I check the Server 2 and 3 for any logs, errors, utilization etc. and they all seems fine except the CPU usage of cd_broker. If we re-start this, the issue got resolved but even in that case the responses are significantly slower than Server 1.

I am referring THIS questions and suggestions provided here and will follow them, but the real question is that why only server 2 and 3 are creating problem and not server 1 since they all are using a centralized cluster of Broker DB?

Few finding:

  • Lots of custom code in the form of batch jobs, broker API job etc. is there
  • I have been told that DB maintenance, Tridion Re-Indexing and other maintenance related tasks are done periodically
  • Nothing important is in the Event Logs

Can someone share his experience or knowledge what else should be checked.

share|improve this question
    
Is this windows or Linux? SQL Server or Oracle? –  Julian Wraith Mar 12 at 12:17
    
Windows Server 2003 SP2 with SQL Server 2005 –  Pankaj Gaur Mar 12 at 12:29

3 Answers 3

up vote 6 down vote accepted

There are quite a few things that can be checked. Its clear that there is a difference in the servers themselves or the way they are operating. Assuming they are just the same specs then I would look at:

  1. The deployed application; are all the files the right versions
  2. The configuration; caching on in all places? Is the config the same?
  3. Is the network on the servers configured the same? Auto vs 1GB
  4. Is there any differences in the load the servers are taking, load balancer issues
  5. Assume that all the servers have resources left (memory, cpu)?
share|improve this answer
    
Thanks Jule!! All are good points from 1 to 4. Regarding 5, during the outage, the CPU and Memory utilization oscillate a lot. I will get verification on 1 and 4 and update my findings –  Pankaj Gaur Mar 12 at 12:33
    
Hey Jule, Regarding (3), Network is configured as Auto on all servers. Regarding (2) since Deployer is not set on Server 2 & 3, publishing is happening up on only server 1, how should CCS supposed to work - in RemoteSynchronization tag, should it point to Server 1 or localhost (server 2) itself? Just wanted to confirm –  Pankaj Gaur Mar 13 at 9:42

I would be suspicious of DFS. If it is copying the files too often, you will trigger unnecessary recompiles, and therefore application restarts.

You can investigate this by using perfmon and adding some ASP.NET counters, specifically CompilationsTotal and ApplicationRestarts. Then just check if the results correlate with your performance dropouts.

share|improve this answer
    
Thanks Dom, I will try this and update my findings :) –  Pankaj Gaur Mar 12 at 12:30

Thanks to both- Dominic and Julian for giving various suggestions based on which finally we are able to settle this issue. There were quite a few issues; first as per suggestion from Dom and Jules, we have used the windows performance monitor to figure out connection to IIS and identified that the load is not balanced evenly on all servers, also the server which was responding very well seems to be out of LB as not receiving any hit on it at all.

Apart from it, we have also noticed that another issue was caused by the DFS as along with it a Full Backup task also get executed suppressing the performance on the primary server and on all others.

Thanks again Dom and Jules and I will throw a dice to identify whose answer should I accept :) as both these answers helps me tremendously.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.