Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16247

SparkSQL Avro serialization doesn't handle enums correctly

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.0.0
    • None
    • hbase-connectors, spark
    • None

    Description

      Avro's generic api expects GenericEnumSymbol as the runtime type for instances of fields that are of Avro type ENUM. The Avro 1.7 libraries are lax in some cases for handling this, but the 1.8 libraries are strict. We should proactively fix our serialization.

      (the lax serialization in 1.7 fails for some nested use in unions, see AVRO-997 for details)

      Attachments

        Issue Links

          blocks

          Sub-task - The sub-task of the issue HBASE-16242 Upgrade Avro to 1.7.7

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-16246 Ensure avro version in hbase-spark matches that shipped in our default hadoop profile

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              Unassigned Unassigned
              busbey Sean Busbey
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: