Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35918

Consolidate logic between AvroSerializer/AvroDeserializer for schema mismatch handling and error messages

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.2
    • 3.3.0
    • SQL
    • None

    Description

      While working on PR #31490 for SPARK-34365, we discussed that there is room for improvement in how schema mismatch errors are reported (comment1, comment2). We can also consolidate more of the logic between AvroSerializer and AvroDeserializer to avoid some duplication of error handling and consolidate how these error messages are generated.

      This will essentially be taking the logic from the initial proposal from PR #31490, but applied separately from the current set of proposed changes to cut down on PR size.

      Attachments

        Issue Links

          Activity

            People

              xkrogen Erik Krogen
              xkrogen Erik Krogen
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: