[GOBBLIN-154] SimpleJsonConverter raises ClassCastException when long type used in schema

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: Bug:Generic Converter:JsonToAvro question
• External issue URL: https://github.com/linkedin/gobblin/issues/433
• Assignee: Unassigned
• Reporter: Abhishek Tiwari
• Created: 19/Jul/17 07:40
• Updated: 19/Jul/17 07:40

Description

I trying to run the given example source/converter named SimpleJsonConverter to run with our own json/schema.

Part of schema looks like this:

{name: tag, type: [long, null]}

And an example row from the source data:
tag:

{long:1}

,

When i run the job it raises a cast exception, i figured out because gson treats each number as double precision (double); when the target object does not specify the expected type, and since that is a generic Map<String, Object> it becomes a Double. See SimpleJsonConverter::convertRecord.

The converter should be generic, i.e. supports any schema given; i'm not sure how to tell gson to apply the right parsing based on the schema?

Caused by: java.lang.ClassCastException: java.lang.Double cannot be cast to java.lang.Long
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:79)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:114)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:104)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:66)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:58)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:290)

Github Url : https://github.com/linkedin/gobblin/issues/433
Github Reporter : Paul424
Github Created At : 2015-11-05T08:17:12Z
Github Updated At : 2017-01-12T04:29:55Z