Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
I had a field whose type changed from ["null", "int"] to ["null", "string"]. When reading a file with the old type (an int) in it, the Avro reader gave this error message:
org.apache.avro.AvroTypeException: Found "int", expecting ["null","string"]
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:212)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:187)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:125)
at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:149)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:121)
at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:112)
at org.apache.avro.file.DataFileStream.next(DataFileStream.java:198)
at org.apache.avro.mapred.AvroRecordReader.next(AvroRecordReader.java:67)
at org.apache.avro.mapred.AvroRecordReader.next(AvroRecordReader.java:34)
It would be helpful if the generate method had context about what record and field name it was working on, so the message could be
org.apache.avro.AvroTypeException: Found "int", expecting ["null","string"] in field "my_field_name" in record "record_name"
I think this would mean adding a field and a record parameter to the ResolvingGrammarGenerator.generate and resolve* methods so an error could identify where it came from.
E.g., the data about the field was available in the resolveRecords methods in this trace of where that error got generated.
Symbol.error(String) line: 122
ResolvingGrammarGenerator.generate(Schema, Schema, Map<LitS,Symbol>) line: 170
ResolvingGrammarGenerator.resolveUnion(Schema, Schema, Map<LitS,Symbol>) line: 186
ResolvingGrammarGenerator.generate(Schema, Schema, Map<LitS,Symbol>) line: 120
ResolvingGrammarGenerator.resolveRecords(Schema, Schema, Map<LitS,Symbol>) line: 254