Japanese characters on SDL Tridion GUI replaced with '?' on IIS after publishing

Question

I am using Japanese characters in my SDL Tridion publication .These characters are visible on SDL Tridion GUI but are replaced with '?' mark after publishing on IIS server. Is there any way to resolve this problem. I am using core service to create or publish the publication ?

Answer 1

Please check following

• Publication Target is set to UTF-8

• Delivery weibsite's web.config has following

<globalization fileEncoding="UTF-8" requestEncoding="UTF-8" responseEncoding="UTF-8"/>

Answer 2

Unexpected characters almost always is an encoding issue, and those are very common unfortunately since by default SDL Tridion sets everything to Windows Default, which is CP1252.

To change the encoding to UTF-8, you need to make sure that it is set through the entire publishing flow. So starting with the Publication Target, then the Publisher, Transport Service, Deployer, Broker and ending with your Web Application.

Elena Serghie wrote a nice blogpost about it http://elenaserghie.blogspot.nl/2012/01/7-clues-to-solve-character-encoding.html

I've added the following comment since that is the one I always seem to forget:

Correct file encoding for a Deployer on a Windows Server is done by setting the -DFile.encoding via a jvmarg in the registry. It is the key HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Tridion\Content Delivery\General and this image shows an example of setting it to UTF-8:

Another option is to use the jvm.xml in your Content Delivery config folders, containsing something like:

<!--
This file can be used to send application specific JVM arguments.
-->
<VirtualMachine>
  <Options>
    <!--
    Add the Option tag to define specific JVM options.
    The Option tag can define the MaximumHeapSizeInMB (-Xmx), the InitialHeapSizeInMB (-Xms) and other JVM properties(-D).
    -->
    <!-- 
    <Option Name="-Xmx" Value="400m"/>
    <Option Name="-Xms" Value="50m"/>
    -->
    <Option Name="-Dfile.encoding" Value="UTF-8"/>
  </Options>
</VirtualMachine>

Since I found that most of the issues seem to relate to incorrect file encoding of the JVM for both the Deployer and the Broker. So do remember to set these for all your Deployer nodes.