[HIVE-11713] Map instances with null keys are not written unless table file format is ORC.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.14.0, 1.2.1
• Component/s: Hive
• Assignee: Unassigned
• Reporter: Takahiko Saito
• Created: 01/Sep/15 22:14
• Updated: 27/Oct/15 20:35

Description

Similar to HIVE-11625, but the issue is seen not only for Parquet, but also SEQUENCEFILE, TEXTFILE, RCFILE:

In a case of ORC fileformatted table, null-key map instance is written to table as expected.

hive> CREATE TABLE map_test stored as ORC AS SELECT MAP(null, 'foo', 1, 'bar', null, 'baz');
Query ID = hrt_qa_20150901220729_a352aa55-78be-4dd7-87d5-ae43811175ad
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1441044573138_0056)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 6.55 s
--------------------------------------------------------------------------------
Moving data to: hdfs://os-d7-0831-hs2concur-r-1.novalocal:8020/apps/hive/warehouse/map_test
Table default.map_test stats: [numFiles=1, numRows=1, totalSize=351, rawDataSize=178]
OK
Time taken: 8.295 seconds
hive> select * from map_test;
OK
{null:"baz",1:"bar"}
Time taken: 0.136 seconds, Fetched: 1 row(s)

However, other fileformatted table, for example, textfile, null-key map instance is not written:

hive> CREATE TABLE map_test AS SELECT MAP(null, 'foo', 1, 'bar', null, 'baz');
Query ID = hrt_qa_20150901220929_e420f3a4-ed0c-472d-8d27-89cca95b537e
Total jobs = 1
Launching Job 1 out of 1


Status: Running (Executing on YARN cluster with App id application_1441044573138_0056)

--------------------------------------------------------------------------------
        VERTICES      STATUS  TOTAL  COMPLETED  RUNNING  PENDING  FAILED  KILLED
--------------------------------------------------------------------------------
Map 1 ..........   SUCCEEDED      1          1        0        0       0       0
--------------------------------------------------------------------------------
VERTICES: 01/01  [==========================>>] 100%  ELAPSED TIME: 6.12 s
--------------------------------------------------------------------------------
Moving data to: hdfs://os-d7-0831-hs2concur-r-1.novalocal:8020/apps/hive/warehouse/map_test
Table default.map_test stats: [numFiles=1, numRows=1, totalSize=13, rawDataSize=12]
OK
Time taken: 7.618 seconds
hive> select * from map_test;
OK
{1:"bar"}
Time taken: 0.118 seconds, Fetched: 1 row(s)

Issue Links

• relates to: HIVE-5020 HCat reading null-key map entries causes NPE