[DRILL-1999] Drill should expose the Parquet logical schema rather than the physical schema

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Component/s: Storage - Parquet
• Assignee: Igor Guzenko
• Reporter: Ramana Inukonda Nagaraj
• Created: 13/Jan/15 20:16
• Updated: 21/Oct/19 09:20
• Resolved: 21/Oct/19 09:20

Description

Created a parquet file in hive having the following DDL
hive> desc alltypesparquet;
OK
c1 int
c2 boolean
c3 double
c4 string
c5 array<int>
c6 map<int,string>
c7 map<string,string>
c8 struct<r:string,s:int,t:double>
c9 tinyint
c10 smallint
c11 float
c12 bigint
c13 array<array<string>>
c15 struct<r:int,s:struct<a:int,b:string>>
c16 array<struct<m:map<string,string>,n:int>>
Time taken: 0.076 seconds, Fetched: 15 row(s)

column5 which is an array of integers shows up as a bag when querying through drill
0: jdbc:drill:> select c5 from `/user/hive/warehouse/alltypesparquet`;
------------

c5

------------

{"bag":[]}

{"bag":[]}

{"bag":[{"array_element":1},{"array_element":2}]}

------------
3 rows selected (0.085 seconds)

While from hive
hive> select c5 from alltypesparquet;
OK
NULL
NULL
[1,2]

Issue Links

• is related to: DRILL-2000 Hive generated parquet files with maps show up in drill as map(key value)