Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-2720

Avro should display invalid field data on AvroTypeException

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.11.0
    • Component/s: python
    • Labels:
      None

      Description

      Calling "Validate"/"write" raises an AvroTypeException when the input "datum" does not fit the input "expected_schema". It would be useful for users if AvroTypeException specified which fields were not valid representations of the expected_schema, by adding additional information to the string printed when the error is thrown. 

       

       

      writer_schema = schema.parse("""\
           {"type": "record", "name": "Test",
           "fields": [{"name": "F", "type": "int"},
           {"name": "E", "type": "int"}]}""")
       datum_to_write = {'E': 5, 'F': 'Bad'}
      
       datum_writer = avro_io.DatumWriter(writer_schema)
       datum_writer.write(datum, encoder)
      

       

      The last line of the above example ^ will throw an AvroTypeException, because field "F" isn't an int. If there are multiple fields with incompatible types, the message will only show the first field. Once it is fixed, it will throw another AvroTypeException with the second incompatible type field, and so on. 

       

      Current AvroTypeException Output:

      avro.io.AvroTypeException: The datum {'E': 5, 'F': 'Bad'} is not an example of the schema {
           "type": "record",
           "name": "Test",
           "fields": [{"type": "int", "name": "F"}, {"type": "int", "name": "E"}]}

      Suggested AvroTypeException Output:

       

      avro.io.AvroTypeException: Field 'F' with value 'Bad' has an unexpected type. The datum {'E': 5, 'F': 'Bad'} is not an example of the schema {
          "type": "record",
          "name": "Test",
          "fields": [{"type": "int", "name": "F"}, {"type": "int", "name": "E"}]}
      

       

        Attachments

          Activity

            People

            • Assignee:
              subhashb Subhash Bhushan
              Reporter:
              chelseadole Chelsea Dole

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment