[HIVE-14006] Hive query with UNION ALL fails with ArrayIndexOutOfBoundsException

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Component/s: Hive
• Assignee: Naveen Gangam
• Reporter: Naveen Gangam
• Created: 13/Jun/16 20:32
• Updated: 21/Jun/16 15:06
• Resolved: 16/Jun/16 15:16

Description

set hive.cbo.enable=false;
DROP VIEW IF EXISTS a_view;

DROP TABLE IF EXISTS table_a1;
DROP TABLE IF EXISTS table_a2;
DROP TABLE IF EXISTS table_b1;
DROP TABLE IF EXISTS table_b2;

CREATE TABLE table_a1
(composite_key STRING);

CREATE TABLE table_a2
(composite_key STRING);

CREATE TABLE table_b1
(composite_key STRING, col1 STRING);

CREATE TABLE table_b2
(composite_key STRING);

CREATE VIEW a_view AS
SELECT
substring(a1.composite_key, 1, locate('|',a1.composite_key) - 1) AS autoname,
NULL AS col1
FROM table_a1 a1
FULL OUTER JOIN table_a2 a2
ON a1.composite_key = a2.composite_key
UNION ALL
SELECT
substring(b1.composite_key, 1, locate('|',b1.composite_key) - 1) AS autoname,
b1.col1 AS col1
FROM table_b1 b1
FULL OUTER JOIN table_b2 b2
ON b1.composite_key = b2.composite_key;

INSERT INTO TABLE table_b1
SELECT * FROM (
SELECT 'something|awful', 'col1'
)s ;

SELECT autoname
FROM a_view
WHERE autoname='something';

fails with
Diagnostic Messages for this Task:
Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

{"_col0":"something"}

at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:179)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row

{"_col0":"something"}

at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:507)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:170)
... 8 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 0
at org.apache.hadoop.hive.ql.exec.UnionOperator.processOp(UnionOperator.java:134)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:815)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.MapOperator$MapOpCtx.forward(MapOperator.java:157)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:497)

The same query succeeds when hive.ppd.remove.duplicatefilters=false with or without CBO on. It also succeeds with just CBO on.