[DRILL-2005] Create table fails to write out a parquet file created from hive- read works fine

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 1.0.0
• Component/s: Storage - Writer
• Assignee: Abdel Hakim Deneche
• Reporter: Ramana Inukonda Nagaraj
• Created: 13/Jan/15 22:49
• Updated: 15/May/15 20:13
• Resolved: 15/May/15 20:13

Description

Created a parquet file in hive having the following DDL
hive> desc alltypesparquet;
OK
c1 int
c2 boolean
c3 double
c4 string
c5 array<int>
c6 map<int,string>
c7 map<string,string>
c8 struct<r:string,s:int,t:double>
c9 tinyint
c10 smallint
c11 float
c12 bigint
c13 array<array<string>>
c15 struct<r:int,s:struct<a:int,b:string>>
c16 array<struct<m:map<string,string>,n:int>>
Time taken: 0.076 seconds, Fetched: 15 row(s)

Now tried to write the file out using drill
0: jdbc:drill:> create table hive_alltypesparquet as select * from `/user/hive/warehouse/alltypesparquet`;
Query failed: Query failed: Failure while running fragment., Attempted to access index 2147483646 when value capacity is 4095 [ 7c030695-9dee-4e15-b9a7-7c807dbd3e5f on 10.10.30.167:31010 ]
[ 7c030695-9dee-4e15-b9a7-7c807dbd3e5f on 10.10.30.167:31010 ]

Drill is able to read the file (reasonably well- see 1997,1999,2000)
0: jdbc:drill:> select * from `/user/hive/warehouse/alltypesparquet`;
--------------------------------------------------------------------------------------------------------------------------------------------------------

c1

c2

c3

c4

c5

c6

c7

c8

c9

c10

c11

c12

c13

c15

c16

--------------------------------------------------------------------------------------------------------------------------------------------------------

null

null

null

null

{"bag":[]}

{"map":[]}

{"map":[]}

{}

null

null

null

null

{"bag":[]}

{"s":{}}

{"bag":[]}

-1

false

-1.1

[B@62369833

{"bag":[]}

{"map":[]}

{"map":[]}

{}

-1

-1

-1.0

-1

{"bag":[]}

{"s":{}}

{"bag":[]}

1

true

1.1

[B@6e426ea4

{"bag":[{"array_element":1},{"array_element":2}]}

{"map":[{"key":1,"value":"eA=="},{"key":2,"value":"eQ=="}]}

{"map":[{"key":"aw==","value":"dg=="}]}

{"r":"YQ==","s":9,"t":2.2}

1

1

1.0

1

{"bag":[{"array_element":{"bag":[{"array_element":"YQ=="},{"array_element":"Yg=="}]}},{"array_element":{"bag":[{"array_element":"Yw=="},{"array_element":"ZA=="}]}}]}

{"r":1,"s":{"a":2,"b":"eA=="}}

Unknown macro: {"bag"}

--------------------------------------------------------------------------------------------------------------------------------------------------------
3 rows selected (0.101 seconds)

Issue Links

• relates to: DRILL-2366 Query nested lists causing IllegalArgumentException