[SLING-5973] HTMLSerializer not handling some unicode characters (emoji, etc.)

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Extensions
• Assignee: Unassigned
• Reporter: Ben Fortuna
• Created: 18/Aug/16 07:21
• Updated: 20/Oct/16 13:12

Description

I've noticed that when I have unicode special characters (e.g. emoji) in my sling content and the sling rewriter is enabled the characters are not output correctly to the browser. For example:

😁

becomes

��

If I disable the rewriter pipeline the output is as expected.

I've looked in the code and I suspect the issue is in the HTMLSerializer from the Cocoon library, however I'm not sure why as it should be using the default encoding for output (which is UTF-8). My rewriter pipeline is using the default html-generator and html-serializer provided by sling.

My code is available on GitHub here:

https://github.com/Whistlepost/emojistrip

It provides a very simple app/content project pair with some emoji characters in the content (see src/main/resources/SLING-INF/content/phrases.json). Many thanks.

Issue Links

• relates to: COCOON-2352 XMLEncoder doesn't support Unicode surrogate pairs