Chinese encoding issue in Broker

Question

Have a Chinese encoding issue on my DD4T site:

My MVC application already supports the Chinese language for another site - Hong Kong, so I'm pretty confident its not an ASP.NET-side problem.

For this site though, we used new AWS China based kit - so a new Deployer server, new server with SQL Server installed and new server to host the site. We then created a new publication target to this Deployer, linked it to this site's publication and published away!

Its then that we saw the encoding issue.

As we have new kit, I'm suspicious that a setting isn't right. I've double checked everything in Tridion. I checked the package being sent to the Deployer - everything looks OK:

But when I retreive the DCP through the Tridion API, the encoding issue is there:

Does this mean its being stored in the SQL Server with the incorrect encoding?

Are there any JVM settings I can use in case its the JVM - either on the Deployer side or the API side?

UPDATE

Also tried with LINQPad against the new Broker db of the affected application - encoding issue was visible:

For my money its got to be either the Deployer's JVM - which is putting stuff in to the Broker db, the SQL Server database itself - which could be storing it wrong, or the API's JVM which is pulling it out wrong?

After running the same LINQ query against my other Chinese language site's Broker (a different one), I can see everything is as expected:

This has got to be JVM or SQL Server right? How can I narrow down my search for the problem?

Answer 1

Ok - so I've come across this setting before but for some reason I assumed it didn't apply. I stuck it in and low and behold, encoding back to normal!

I simply added a new System Variable JAVA_TOOL_OPTIONS under Environmental Variables on the Deployer server and gave it the value:

-Dfile.encoding=UTF8

Restarted IIS, republished and BOOM!

Answer 2

Is this a new Publication Target as well as Deployer, Broker DB etc?

If so, have you checked the Default Code Page setting for the new Target matches the working one?

That's the first step in Elena's (in)famous "7 Clues to Solve Character Encoding Issues" blog post. I don't want to re-write her article here, but I'd have a read through that for other possibilities.

Answer 3

Looking at the additional information in your question update I see that you can see differences in the stored content against your existing broker DB. In this case if it's not the Publication Target's settings I'd definitely look at the Deployer's JVM configuration.