[HIVE-8909] Hive doesn't correctly read Parquet nested types

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.13.1
• Fix Version/s: 1.1.0
• Assignee: Ryan Blue
• Reporter: Ryan Blue
• Created: 19/Nov/14 00:46
• Updated: 11/Dec/17 03:02
• Resolved: 23/Nov/14 17:54

Description

Parquet's Avro and Thrift object models don't produce the same parquet type representation for lists and maps that Hive does. In the Parquet community, we've defined what should be written and backward-compatibility rules for existing data written by parquet-avro and parquet-thrift in PARQUET-113. We need to implement those rules in the Hive Converter classes.

Issue Links

• blocks: PARQUET-83 Hive Query failed if the data type is array<string> with parquet files
• breaks: HIVE-9502 Parquet cannot read Map types from files written with Hive <= 0.12
• is depended upon by: HIVE-6914 parquet-hive cannot write nested map (map value is map)
• is depended upon by: PARQUET-26 Parquet doesn't recognize the nested Array type in MAP as ArrayWritable.
• is depended upon by: HIVE-8949 Test for PARQUET-26
• is related to: HIVE-18211 Support to read multiple level definition for Map type in Parquet file