[HIVE-14736] It doesn't work WITH and UNION ALL

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Ryu Kobayashi
• Created: 12/Sep/16 06:07
• Updated: 27/Jul/18 02:51

Description

When a combination of WITH and UNION ALL, may not work.
example:

with a as (
  select col1, col2 from table
), b as (
  select col1 from a where col2 is not null
), c as (
  select col1 from a
)
select col1 from b
union all
select col1 from c

Exception:

org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: Hive Runtime Error while closing operators
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.NullPointerException
	at org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1032)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:682)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
	at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:696)
	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
	... 8 more

      for (FSPaths fsp : valToPaths.values()) {
        fsp.closeWriters(abort);
        // before closing the operator check if statistics gathering is requested
        // and is provided by record writer. this is different from the statistics

This is caused when the #initializeOp is not initialized.

Issue Links

• relates to: HIVE-9832 Merge join followed by union and a map join in hive on tez fails.