Intermittent broken DXA JSON output during publishing

Question

Below are our environment and web application details:

SDL Tridion 2013 SP1 HR1, Oracle DB, DXA1.2

We are facing an intermittent error “exception while deserializing into CP” in the publisher log in PROD whenever there is an high load.

I have raised a question about this already DXA 1.2 intermittent issue in JSON output and got feedback that upgrade would fix the issue. But, I see the same issue with DXA1.5 as posted here DXA 1.5 intermittent issue in JSON output

To explore this issue more, we have started to download/build and debug the DXA 1.2 content management solutions by adding more logs in the following classes.

1. “DD4T.Templates.Base.Builder.ComponentPresentationBuilder.cs”,
2. “DD4T.Serialization.SerializerServiceFactory.cs”,
3. “DD4T.Serialization.JSONSerializerService.cs”.

To debug/add more logs we have followed the below steps to get the solution up and running:

1. Downloaded the DXA1.2 CM solutions from here. https://github.com/sdl/dxa-content-management/tree/release/1.2
2. We checked this page https://github.com/sdl/dxa-content-management/tree/release/1.2/_references/dd4t-update and found that DD4T used in DXA1.2 CMS side is from here https://github.com/sdl/DD4T.TridionTemplates/tree/feature/ecl-data and downloaded the DD4T CM solution as well.
3. We wanted to debug the “DD4T.Model” project as well. So, we have downloaded the “DD4T.Model” project from here https://github.com/sdl/DD4T.Model/tree/DXA_1.2_Release "The reason why we didn’t downloaded the “2.0.4-beta1” is this does not have the updates to deal the “ECL” which is required in the “DD4T ecl” branch “EclProcessor.cs” class"
4. Now, after downloading all the above 3 solutions, we are able to build and compile the solutions without any issue.
5. To create the “DD4T.Templates.merged.dll”, we have followed the steps from here. DD4T - Generate DD4T.Templates.merged.dll using ILMerge manually

Question:

1. Can anyone confirm the GitHub repositories that were used to download the solutions are correct?

2. Also, can anyone confirm the steps that were recommended to create the “DD4T.Templates.merged.dll” are correct? Especially the “DLL’s” order and no of input DLL’s.

   a. The reason behind this question is to avoid any merge issue when we are adding more custom logs in a working PROD environment.

3. When we check this class https://github.com/dd4t/DD4T.Model/blob/master/source/DD4T.Serialization/JSONSerializerService.cs we found that there is an fix for the concurrency issue. We don’t have that fix in the above mentioned solution’s and in our PROD (decompiled the dll and checked that). Will this fix also potentially resolve the intermittent issue “exception while deserializing into CP”?

   a. This concurrency fix is done in the “DD4T” repository not in the “SDL” repository. But we have downloaded all the 3 solutions from the “SDL” repository. So, we are not sure we can have this fix in our downloaded code and not sure whether this will fix the deserializing error. Any thoughts on this?

   b. Also, we have opened an issue here https://github.com/dd4t/DD4T.Model/issues/35

Note:

• When I decompile the dll “DD4T.Templates.merged.dll” from PROD, I found that the dll version is “2.0.4.0”. I understand that the first input DLL (DD4T.ContentModel.Contracts.dll) version will be carried to merged dll.
• But in the above DD4T ecl branch solution the referred “DD4T.Model” version is “2.0.5.19-alpha”. So to get the closest DD4T.Model project, we have downloaded the DD4T.Model solution from here.

Answer 1

As I commented on your original question already, I don't think an upgrade to a DXA 1.x version will solve the issue you are dealing with, since there haven't been that much changes in the DD4T template version we are using there, and the issue you see is something which originates from there.

Having said that, I should probably also mention that we are currently finalizing the DXA 2.0 release and that will use the DXA R2 data model (as showcased in the DXA 2.0 CTP releases) which should not have these issues. But of course I can't be certain, since we don't exactly know yet where it is coming from. We are doing load tests as a standard part of the release builds, but we don't focus a lot on publishing load, (other than publishing the entire DXA example site, which we do multiple times a day).

To focus on answering your questions:

1. You seem to have done quite a thorough job of figuring out which repositories and branches to use, I don't have much to comment on there.

2. I don't believe the order of the DLLs are important for correctly merging, a different order, might deliver you a different final version, but since you are making a custom build, that isn't that important anyways. You will find out quickly enough if you have all the right DLLs for the merge, since that will fail, and even if that succeeds, you might still encounter issues during the upload into the CMS. But I wouldn't worry too much about that, just try the build/merge and try to upload it in your development environment and if that works, it should also work in your production environment.

3. I can't confirm the fix done in DD4T will solve your issue, since I have not been able to reproduce your issue. It looks like it might be a solution, so I advise you to merge that fix into your code base and build with that. Else what you are doing is just rebuilding a DLL that won't be any different than the one you had before, and that certainly won't solve your problem.