Avro
  1. Avro
  2. AVRO-690

Schema resolution error message for union lacks field name

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: java
    • Labels:
      None

      Description

      I had a field whose type changed from ["null", "int"] to ["null", "string"]. When reading a file with the old type (an int) in it, the Avro reader gave this error message:

      org.apache.avro.AvroTypeException: Found "int", expecting ["null","string"]
      at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:212)
      at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
      at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:187)
      at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:125)
      at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:149)
      at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:121)
      at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:112)
      at org.apache.avro.file.DataFileStream.next(DataFileStream.java:198)
      at org.apache.avro.mapred.AvroRecordReader.next(AvroRecordReader.java:67)
      at org.apache.avro.mapred.AvroRecordReader.next(AvroRecordReader.java:34)

      It would be helpful if the generate method had context about what record and field name it was working on, so the message could be

      org.apache.avro.AvroTypeException: Found "int", expecting ["null","string"] in field "my_field_name" in record "record_name"

      I think this would mean adding a field and a record parameter to the ResolvingGrammarGenerator.generate and resolve* methods so an error could identify where it came from.

      E.g., the data about the field was available in the resolveRecords methods in this trace of where that error got generated.

      Symbol.error(String) line: 122
      ResolvingGrammarGenerator.generate(Schema, Schema, Map<LitS,Symbol>) line: 170
      ResolvingGrammarGenerator.resolveUnion(Schema, Schema, Map<LitS,Symbol>) line: 186
      ResolvingGrammarGenerator.generate(Schema, Schema, Map<LitS,Symbol>) line: 120
      ResolvingGrammarGenerator.resolveRecords(Schema, Schema, Map<LitS,Symbol>) line: 254

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            Thiruvalluvan M. G.
            Reporter:
            Ron Bodkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development