Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-10687

AvroDeserializer fails to deserialize evolved union fields

    Details

    • Target Version/s:

      Description

      Consider the union field:

      union {int, string}
      

      and now this field evolves to

      union {null, int, string}.
      

      Running it through the avro schema compatibility check[1], they are actually compatible which means that the latter could be used to deserialize the data written with former. However the avro deserializer fails to do that. Mainly because of the way it reads the tags from the reader schema and then reds the corresponding data from the writer schema. [2]

      [1] http://pastebin.cerner.corp/31078
      [2] https://github.com/cloudera/hive/blob/cdh5.4.0-release/serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java#L354

        Attachments

        1. HIVE-10687.1.patch
          7 kB
          Swarnim Kulkarni

          Activity

            People

            • Assignee:
              swarnim Swarnim Kulkarni
              Reporter:
              swarnim Swarnim Kulkarni
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: