Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8577

Cannot deserialize Avro schema with a map<string,string> with null values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.15.0
    • 0.14.0
    • None

    Description

      An avro table with a map<string,string> column that contains null values
      cannot be deserialized when running the select statement.

      Create the following table:

      CREATE TABLE avro_table (avreau_col_1 map<string,string>) 
      ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS
      INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' 
      OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' 
      TBLPROPERTIES ('avro.schema.url'='file:///tmp/map_null_schema.avro');
      

      Then load the avro data:

      LOAD DATA LOCAL INPATH '/tmp/map_null_val.avro' OVERWRITE INTO TABLE avro_table;
      

      And do the select (it fails):

      SELECT * FROM avro_table;
      Error: java.io.IOException: org.apache.avro.AvroRuntimeException: Not a map: "null" (state=,code=0)
      

      This is a regression bug (it works correctly on hive 0.13.1 version).
      This is the output that hive 0.13.1 displays:

      {"key3":"val3","key4":null}
      {"key3":"val3","key4":null}
      {"key1":null,"key2":"val2"}
      {"key3":"val3","key4":null}
      {"key3":"val3","key4":null}
      

      Attachments

        1. HIVE-8577.2.patch
          7 kB
          Sergio Peña
        2. HIVE-8577.1.patch
          2 kB
          Gunther Hagleitner
        3. HIVE-8577.1.patch
          2 kB
          Sergio Peña
        4. map_null_val.avro
          0.3 kB
          Sergio Peña
        5. map_null_schema.avro
          0.2 kB
          Sergio Peña

        Issue Links

          Activity

            People

              spena Sergio Peña
              spena Sergio Peña
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: