[HIVE-20318] NullPointerException when union with lateral view

• Type: Bug
• Status: Patch Available
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 2.3.2, 3.1.0, 3.0.0, 3.1.1
• Component/s: Query Processor

Environment

Run on MR, hadoop 2.7.3 and Tez

• Assignee: Ryu Kobayashi Assign to me
• Reporter: Wang Yan
• Due: 06/Aug/18
• Created: 06/Aug/18 15:16
• Updated: 17/Jan/19 04:26

Description

The following sql throws NullPointerException.

This sql is not table/data specific and can be run directly.

WITH 
t1 AS (SELECT 0 AS c1),
t2 AS (
	SELECT 0 AS c1
	FROM (
		SELECT COLLECT_SET('line') AS c2
	     ) t3 lateral VIEW explode(ARRAY("a")) er AS c3
      )
SELECT c1
FROM t1

UNION ALL

SELECT c1
FROM t2

This is the exception.

 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Hive Runtime Error while closing operators
 at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:422)
 at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
 Caused by: java.lang.NullPointerException
 at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
 at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
 at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
 at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
 at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
 at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
 at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
 ... 8 more