[HIVE-8577] Cannot deserialize Avro schema with a map<string,string> with null values

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.15.0
• Fix Version/s: 0.14.0
• Labels: regression
• Assignee: Sergio Peña
• Reporter: Sergio Peña
• Created: 23/Oct/14 18:32
• Updated: 26/Jan/15 22:47
• Resolved: 30/Oct/14 02:51

Description

An avro table with a map<string,string> column that contains null values
cannot be deserialized when running the select statement.

Create the following table:

CREATE TABLE avro_table (avreau_col_1 map<string,string>) 
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' STORED AS
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' 
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat' 
TBLPROPERTIES ('avro.schema.url'='file:///tmp/map_null_schema.avro');

Then load the avro data:

LOAD DATA LOCAL INPATH '/tmp/map_null_val.avro' OVERWRITE INTO TABLE avro_table;

And do the select (it fails):

SELECT * FROM avro_table;
Error: java.io.IOException: org.apache.avro.AvroRuntimeException: Not a map: "null" (state=,code=0)

This is a regression bug (it works correctly on hive 0.13.1 version).
This is the output that hive 0.13.1 displays:

{"key3":"val3","key4":null}
{"key3":"val3","key4":null}
{"key1":null,"key2":"val2"}
{"key3":"val3","key4":null}
{"key3":"val3","key4":null}

Issue Links

• blocks: HIVE-8359 Map containing null values are not correctly written in Parquet files
• is broken by: HIVE-5823 Support for DECIMAL primitive type in AvroSerDe
• is related to: HIVE-9462 HIVE-8577 - breaks type evolution