[HIVE-16118] Some queries with multiple inserts fail in Hive-1.2 and work in Hive-0.13

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.2.0, 2.1.0, 2.1.1
• Component/s: Hive
• Assignee: Unassigned
• Reporter: Aleksey Vovchenko
• Created: 06/Mar/17 09:18
• Updated: 24/Apr/18 15:05

Description

STEPS TO REPRODUCE:

1. Create tables and insert data into it:

CREATE TABLE `table1` (`id` INT, `date` TIMESTAMP);
CREATE TABLE `table2` (`id` INT, `date` TIMESTAMP);
CREATE TABLE `table3` (`id` INT, `date` TIMESTAMP);
CREATE TABLE `table4` (`id` INT, `date` TIMESTAMP);

INSERT OVERWRITE TABLE `table1` VALUES (1,'2006-03-30 19:42:06'),(2,'2014-05-11 09:39:11'),(3,'2010-09-01 04:42:17'),(4,'2012-01-04 19:56:20'),(5,'2011-02-12 03:03:42');
INSERT OVERWRITE TABLE `table2` VALUES (1,'2006-03-30 19:42:06'),(2,'2014-05-11 09:39:11'),(3,'2010-09-01 04:42:17'),(4,'2012-01-04 19:56:20'),(5,'2011-02-12 03:03:42');

2. Run the query with multiple inserts:

FROM `table1` AS `t1`
LEFT OUTER JOIN `table2` AS `t2`
ON `t1`.`id` = `t2`.`id`
INSERT OVERWRITE TABLE `table3`
SELECT `t1`.`id`, `t1`.`date`
WHERE STRING(`t1`.`date`) <= STRING(from_unixtime(unix_timestamp(),'yyyy-MM-dd'))
INSERT OVERWRITE TABLE `table4`
SELECT `t1`.`id`, `t1`.`date`
WHERE `t1`.`id` <= INT(IF(`t2`.`id` IS NULL,0,`t2`.`id`));

ACTUAL RESULT:

The query failed with such exception:

2017-02-22 11:00:03,904 ERROR [Thread-14]: exec.Task (SessionState.java:printError(963)) -
Task with the most failures(4):

---

Task ID:
task_1487688226052_0091_m_000000

---

Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

{"id":1,"date":"2006-03-30 19:42:06"}

at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:172)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:458)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:348)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

{"id":1,"date":"2006-03-30 19:42:06"}

at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:546)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:163)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unexpected exception: org.apache.hadoop.hive.serde2.SerDeException: Error: expecting 2 but asking for field 2
data=[Ljava.lang.Object;@39836856
tableType=struct<id:int,date:timestamp>
dataType=struct<_col0:int,_col1:timestamp,_col5:int>
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:426)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:97)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:165)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:536)
... 9 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.serde2.SerDeException: Error: expecting 2 but asking for field 2
data=[Ljava.lang.Object;@39836856
tableType=struct<id:int,date:timestamp>
dataType=struct<_col0:int,_col1:timestamp,_col5:int>
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:789)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
at org.apache.hadoop.hive.ql.exec.FilterOperator.process(FilterOperator.java:122)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:838)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.internalForward(CommonJoinOperator.java:644)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.genAllOneUniqueJoinObject(CommonJoinOperator.java:676)
at org.apache.hadoop.hive.ql.exec.CommonJoinOperator.checkAndGenObject(CommonJoinOperator.java:754)
at org.apache.hadoop.hive.ql.exec.MapJoinOperator.process(MapJoinOperator.java:414)
... 13 more
Caused by: org.apache.hadoop.hive.serde2.SerDeException: Error: expecting 2 but asking for field 2
data=[Ljava.lang.Object;@39836856
tableType=struct<id:int,date:timestamp>
dataType=struct<_col0:int,_col1:timestamp,_col5:int>
at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.doSerialize(LazySimpleSerDe.java:239)
at org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe.serialize(AbstractEncodingAwareSerDe.java:50)
at org.apache.hadoop.hive.ql.exec.FileSinkOperator.process(FileSinkOperator.java:720)
... 20 more

Issue Links

• is related to: HIVE-19286 NPE in MERGE operator on MR mode