Nginx + Confluence Encoding Issues

[Corey McCown]

Throughout the past several versions of Confluence, we've run into an encoding issue on our instance. We are running Confluence Server 6.11.0 currently, this can be updated if needed for troubleshooting. The issues have occurred since at least 6.8.0. 

In the WYSIWYG editor, any widget added has garbled characters. Viewing the widgets on a published page appears correct, the issue is strictly on the page editor. Everything else appears to work properly.

System Details:
CentOS Linux release 7.5.1804 (Core)
Nginx 1.12.2
MySQL 5.6.41
Confluence Language: EN (US)
Confluence, Nginx and MySQL are all on the same server/localhost.

I ensured our collation and character set was correct, per the knowledge base article.

I ran the Character Encoding Test, here are the results:

Test 1: Raw text
This is the test string generated in Confluence

Iñtërnâtiônàlizætiøn

Test 2: Form submission
This is the test string pasted by you into the web form and submitted back to Confluence

Iñtërnâtiônàlizætiøn

Test 3: Database round-trip (select as lower-case)
This is the string from Test 2 after being stored in the database and then retrieved as lower-case

iñtërnâtiônàlizætiøn

Expected result (converting Java string to lowercase)

iñtërnâtiônàlizætiøn

Test 4: Database round-trip (select as upper-case)
This is the string from Test 2 after being stored in the database and then retrieved as upper-case

IÑTËRNÂTIÔNÀLIZÆTIØN

Expected result (converting Java string to uppercase)

IÑTËRNÂTIÔNÀLIZÆTIØN

Test 5: International file name support
Try to write a file to the confluence home directory with the test string as the file name

File was written successfully

Test 6: Detect international file name mangling
Detect whether the file system is mangling the file name when it is saved

The file name has been mangled

As seen, Test 6 fails with "The file name has been mangled." I've done a ton of Googling and tried all of the suggestions I could find, I'm open to any and every suggestion.

Here are our configuration files. I redacted some information, such as the domain, but anything pertinent remains.

/etc/nginx/nginx.conf

http {
 log_format main '$remote_addr - $remote_user [$time_local] "$request" '
 '$status $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';

 access_log /var/log/nginx/access.log main;

 server_tokens off; #Disable version in header
 sendfile on;
 tcp_nopush on;
 tcp_nodelay on;
 keepalive_timeout 65;
 types_hash_max_size 2048;
 client_max_body_size 100M;

 include /etc/nginx/mime.types;
 default_type application/octet-stream;

 # Load modular configuration files from the /etc/nginx/conf.d directory.
 # See http://nginx.org/en/docs/ngx_core_module.html#include
 # for more information.
 include /etc/nginx/conf.d/*.conf;

 server {
 listen 80 default_server;
 listen [::]:80 default_server;
 server_name atlas.xxx.xxx;
 return 301 https://$host$request_uri;
 }

 # HTTPS Proxy
 server {
 listen 443 ssl http2;
 listen [::]:443 ssl http2;
 server_name atlas.xxx.xxx;
 charset UTF-8;
 ssl_certificate "/etc/certs/ssl-bundle.crt";
 ssl_certificate_key "/etc/certs/atlas.xxx.xxx.key";
 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 ssl_prefer_server_ciphers on;
 ssl_ciphers "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA$
 ssl_ecdh_curve secp384r1;
 ssl_session_cache shared:SSL:10m;
 ssl_session_tickets off;
 ssl_dhparam "/etc/certs/dhparam.pem";
 chunked_transfer_encoding off;

 include /etc/nginx/default.d/*.conf;

 location / {
 proxy_pass http://127.0.0.1:8090;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header Host $host;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 }

 location /synchrony {
 proxy_set_header X-Forwarded-Host $host;
 proxy_set_header X-Forwarded-Server $host;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_pass http://127.0.0.1:8091/synchrony;
 proxy_http_version 1.1;
 proxy_set_header Upgrade $http_upgrade;
 proxy_set_header Connection "Upgrade";
 }
 }
}

 /opt/atlassian/confluence/conf/server.xml

<Connector port="8090" connectionTimeout="20000" redirectPort="8443"
 maxThreads="48" minSpareThreads="10" URIEncoding="UTF-8"
 enableLookups="false" acceptCount="10" debug="0"
 protocol="org.apache.coyote.http11.Http11NioProtocol"
 scheme="https" secure="true" proxyName="atlas.xxx.xxx" proxyPort="443"/>

Please let me know if there is any other information you'd like me to provide. Thank you in advanced for any assistance.

EDIT:
After adding the following to my setenv.sh file, all encoding tests pass:

export JAVA_OPTS="$JAVA_OPTS -Djavax.servlet.request.encoding=UTF8 -Dfile.encoding=UTF8"

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LANGUAGE=en_US.UTF-8

 I am still receiving garbled characters on widgets in the WYSIWYG editor:

[Corey McCown · Accepted solution]

Found a related question deep in Google: https://community.atlassian.com/t5/Confluence-questions/unreadable-text-in-the-macro-header/qaq-p/661943

I resolved this issue by installing the following fonts:

yum install dejavu-fonts-common
yum install dejavu-sans-fonts
yum install dejavu-sans-mono-fonts

 Thank you @Lava Kumar Dukanam _Appfire_ for helping resolve the encoding issues - it looks like this was actually due to missing fonts. Glad to have both issues resolved!

-CM

[James Richards]

Hi,

For reference this document has more details

• Confluence UI shows garbled or corrupt text on CAPTCHA, macros and/or diagrams due to missing fonts

https://confluence.atlassian.com/confkb/confluence-ui-shows-garbled-or-corrupt-text-on-captcha-macros-and-or-diagrams-due-to-missing-fonts-938027858.htmlConfluence UI shows garbled or corrupt text on CAPTCHA, macros and/or diagrams due to missing fonts James

[Lava Kumar Dukanam _Appfire_]

Hi @Corey McCown,

Do you have this file.encoding setting in setenv.sh (located under <confluence-home>/confluence/bin) ?

set "JAVA_OPTS=%JAVA_OPTS% -Dfile.encoding=UTF8"

 Let me know if that helps.

You might have to restart your instance once you add it in.

Thanks,

Lava

[Corey McCown]

Hi @Lava Kumar Dukanam _Appfire_,

Thanks for the response. I added in the Dfile.encoding setting and restarted, it still appears to have the same issue and fails the "file name has been mangled" test.

I actually added the following line per StackOverflow:

export JAVA_OPTS="$JAVA_OPTS -Dfile.encoding=UTF8"

 I also tried

export JAVA_OPTS="$JAVA_OPTS -Djavax.servlet.request.encoding=UTF8 -Dfile.encoding=UTF8"

Thanks,

-Corey

[Corey McCown]

I updated my original post. I was able to add a few more exports to the setenv.sh file and resolve the encoding test page. That said, I am still having issues with the widgets having garbled text in the WYSIWYG editor.