[AVRO-1072] The JSON encoder doesn't handle non-ASCII character properly

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.6.3
• Component/s: java

Environment

uname -a
Darwin zmac 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun 7 16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386

java -version
java version "1.6.0_29"
Java(TM) SE Runtime Environment (build 1.6.0_29-b11-402-10M3527)
Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02-402, mixed mode)

• Assignee: Unassigned
• Reporter: Zhihong Zhang
• Created: 26/Apr/12 22:08
• Updated: 08/May/12 21:39

Description

The JSON encoder uses default encoding of the platform. It should always use UTF-8.

This causes multiple problems for us,

1. The text is mangled if sending/receiving machine has different encoding.
2. Some encodings (like Latin-1 or MacRoman) can't handle all characters (like Chinese) and we get ? in the text.
3. The binary encoder (ByteBuffer) doesn't work due to this problem.