[AVRO-860] Invalid JSON when printing out records with unicode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.5.1
• Component/s: java
• Labels: java json unicode
• Assignee: Miki Tebeka
• Reporter: Miki Tebeka
• Created: 14/Jul/11 18:37
• Updated: 25/Sep/12 21:22
• Resolved: 13/Sep/12 20:09

Description

I have an avro file, that when printed returns invalid JSON.
The code for iterating and printing is:

            DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>();
            DataFileReader<GenericRecord> dataFileReader =
                new DataFileReader<GenericRecord>(data, reader);

            while (dataFileReader.hasNext()) {
                System.out.println(dataFileReader.next().toString());
            }

and the relevant JSON snippet is

    "description": "Move™ offers advertisers the opportunity to deliver messages to consumers at a time when consumers are making the biggest purchases of their lives\uMOVE™ OFFERS ADVERTISERS THE OPPORTUNITY TO DELIVER MESSAGES TO CONSUMERS AT A TIME WHEN CONSUMERS ARE MAKING THE BIGGEST PURCHASES OF THEIR LIVES—OR REMODELING, REDECORATING AND MAINTAINING THEIR MOST IMPORTANT ASSETS.or remodeling, redecorating and maintaining their most important assets.",

(The \uMOVE is the problematic part).

However if I do:

                GenericRecord record = dataFileReader.next();
                Utf8 desc = (Utf8)record.get("description");
                System.out.println(desc);

Then I get

Move™ offers advertisers the opportunity to deliver messages to consumers at a time when consumers are making the biggest purchases of their lives—or remodeling, redecorating and maintaining their most important assets.

Issue Links

• duplicates: AVRO-851 Unicode issue in JSON encoding from Java