We have upgraded CMS from Web 8.1 to Web 8.5 couple of months back. We got three CMS nodes behind the F5, We frequently used to see below list of issues and restarting Content Manager Service Host solves the problem and may happen again after 1 week in that node. At least one node per week shows us this behavior.
List of issues:
Unable to see thumbnails of images inside components.
Used to see below error while page previews or version history comparison etc., in CME.
(80040356) Could not load file or assembly '0 bytes loaded from Tridion.ContentManager.Templating, Version=8.5.0.466, Culture=neutral, PublicKeyToken=360aac4d3354074b' or one of its dependencies. An attempt was made to load a program with an incorrect format. Bad IL format.
Bad IL format.
Unable to see thumbnails in CME media repository (which are ECL from S3)
We are not seeing any bad CPU or memory metrics in that particular node at the time of failure.
So i am looking for some guidance on how to investigate this?
Here is the log from event viewer:
Could not load file or assembly '0 bytes loaded from Tridion.ContentManager.Templating, Version=8.5.0.466, Culture=neutral, PublicKeyToken=360aac4d3354074b' or one of its dependencies. An attempt was made to load a program with an incorrect format.
Bad IL format.
Component: Tridion.ContentManager.CoreService
Errorcode: 0
User: NT AUTHORITY\NETWORK SERVICE
StackTrace Information Details:
at System.Reflection.RuntimeAssembly.nLoadImage(Byte[] rawAssembly, Byte[] rawSymbolStore, Evidence evidence, StackCrawlMark& stackMark, Boolean fIntrospection, Boolean fSkipIntegrityCheck, SecurityContextSource securityContextSource)
at System.Reflection.Assembly.Load(Byte[] rawAssembly)
at Tridion.ContentManager.Templating.TemplateUtilities.LoadAssembly(Engine engine, Byte[] assemblyContent, String assemblyFileName)
at Tridion.ContentManager.AssemblyCache.AssemblyCache.<>c__DisplayClass12_0.<GetOrAddAssemblyCacheInfo>b__0(String _)
at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory)
at Tridion.ContentManager.AssemblyCache.AssemblyCache.GetOrAddAssemblyCacheInfo(String script, Func`1 getAssembly, String keyAddition)
at Tridion.ContentManager.AssemblyCache.AssemblyCache.GetInstanceOfType[T](String script, Func`1 generateAssembly, String typeName, Func`2 getType, String keyAddition)
at Tridion.ContentManager.Templating.Assembly.CSharpSourceCodeMediator.Transform(Engine engine, Template template, Package package)
at Tridion.ContentManager.Templating.Engine.ExecuteTemplate(Template template, Package package)
at Tridion.ContentManager.Templating.Engine.InvokeTemplate(Package package, TemplateInvocation templateInvocation, Template template)
at Tridion.ContentManager.Templating.Compound.CompoundTemplateMediator.Transform(Engine engine, Template templateToTransform, Package package)
at Tridion.ContentManager.Templating.Engine.ExecuteTemplate(Template template, Package package)
at Tridion.ContentManager.Templating.Engine.InvokeTemplate(Package package, TemplateInvocation templateInvocation, Template template)
at Tridion.ContentManager.Templating.Engine.TransformPackage(Template template, Package package)
at Tridion.ContentManager.Templating.Engine.TransformItem(Template template, IdentifiableObject itemToRender)
at Tridion.ContentManager.Templating.TemplatingRenderer.Render(ResolvedItem resolvedItem, PublishInstruction instruction, PublicationTarget target, RenderedItem renderedItem, RenderContext renderContext)
at Tridion.ContentManager.Publishing.Rendering.RenderEngine.Render(ResolvedItem resolvedItem, PublishInstruction instruction, PublicationTarget target, RenderContext context)
at Tridion.ContentManager.Publishing.Rendering.RenderEngine.Render(IdentifiableObject item, Template template, PublishInstruction instruction, PublicationTarget target, RenderContext context)
at Tridion.ContentManager.Publishing.Rendering.RenderEngine.Render(IdentifiableObject item, Template template, PublishInstruction instruction, PublicationTarget target)
at Tridion.ContentManager.CoreService.CoreServiceBase.RenderItem(TcmUri itemId, IdentifiableObjectData itemDeltaData, TcmUri templateId, TemplateData templateDeltaData, PublishInstructionData publishInstruction, String publicationTargetIdOrPurpose)
at Tridion.ContentManager.CoreService.CoreServiceBase.RenderItem(String itemId, String templateId, PublishInstructionData publishInstruction, String publicationTargetIdOrPurpose)
at SyncInvokeRenderItem(Object , Object[] , Object[] )
at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at Tridion.ContentManager.CoreService.CoreServiceInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc& rpc)
at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)