[DRILL-2000] Hive generated parquet files with maps show up in drill as map(key value)

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Component/s: Storage - Parquet
• Assignee: Bohdan Kazydub
• Reporter: Ramana Inukonda Nagaraj
• Created: 13/Jan/15 20:22
• Updated: 19/Sep/19 11:03
• Resolved: 19/Sep/19 10:57

Description

Created a parquet file in hive having the following DDL
hive> desc alltypesparquet;
OK
c1 int
c2 boolean
c3 double
c4 string
c5 array<int>
c6 map<int,string>
c7 map<string,string>
c8 struct<r:string,s:int,t:double>
c9 tinyint
c10 smallint
c11 float
c12 bigint
c13 array<array<string>>
c15 struct<r:int,s:struct<a:int,b:string>>
c16 array<struct<m:map<string,string>,n:int>>
Time taken: 0.076 seconds, Fetched: 15 row(s)

Columns which are maps such as c6 map<string,string>
show up as

0: jdbc:drill:> select c6 from `/user/hive/warehouse/alltypesparquet`;
------------

c6

------------

{"map":[]}

{"map":[]}

{"map":[{"key":1,"value":"eA=="},{"key":2,"value":"eQ=="}]}

------------
3 rows selected (0.078 seconds)

hive> select c6 from alltypesparquet;
NULL
NULL

{1:"x",2:"y"}

Ignore the wrong values, I have raised DRILL-1997 for the same.

Issue Links

• is part of: DRILL-7096 Develop vector for canonical Map<K,V>
• relates to: DRILL-7096 Develop vector for canonical Map<K,V>
• relates to: DRILL-3290 Hive Storage: Add support for Hive complex types
• relates to: DRILL-1999 Drill should expose the Parquet logical schema rather than the physical schema