[AVRO-2149] Using GenericDatumWriter for writing GenericRecord with schema from the specified generated type

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.8.2
• Component/s: java
• Flags: Patch
• Assignee: Unassigned
• Reporter: Wojciech Sroka
• Created: 22/Feb/18 07:41
• Updated: 22/Feb/18 16:32

Description

Currently using GenericDatumWriter for writing GenericRecord with schema from the specified generated type cause exception when one of fields is enum type:

Exception in thread "main" org.apache.avro.file.DataFileWriter$AppendWriteException: org.apache.avro.AvroTypeException: Not an enum: YELLOW
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at pl.ingbank.kafka.AvroErrorTester.runGenericWriterReaderTest(AvroErrorTester.java:65)
at pl.ingbank.kafka.AvroErrorTester.main(AvroErrorTester.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: org.apache.avro.AvroTypeException: Not an enum: YELLOW
at org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:195)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:128)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:177)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:167)
at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:118)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:75)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 7 more

It is caused by method protected void writeEnum(Schema schema, Object datum, Encoder out) in class org.apache.avro.generic.GenericDatumWriter:

  protected void writeEnum(Schema schema, Object datum, Encoder out) throws IOException {
    if (!data.isEnum(datum)) {
      throw new AvroTypeException("Not an enum: " + datum);
    }
    out.writeEnum(schema.getEnumOrdinal(datum.toString()));
  }

Fix for this is to add additional check does datum is enum type:

protected void writeEnum(Schema schema, Object datum, Encoder out) throws IOException {
  if(datum.getClass().isEnum() && !data.isEnum(datum)){
   throw new AvroTypeException("Not an enum: " + datum);
  }
  out.writeEnum(schema.getEnumOrdinal(datum.toString()));
}

This fix alows to serialize and deserialize generic record with schema from the specified generated type.