Description
When the hive.optimize.union.remove, hive.merge.mapfiles and hive.merge.mapredfiles are turned on, it throws NullPointerException when I do the following queries:
create table inputTbl1(key string, val string) stored as textfile; create table outputTbl1(key string, values bigint) stored as rcfile; load data local inpath '../../data/files/T1.txt' into table inputTbl1; explain insert overwrite table outputTbl1 SELECT * FROM ( select key, count(1) as values from inputTbl1 group by key union all select * FROM ( SELECT key, 1 as values from inputTbl1 UNION ALL SELECT key, 2 as values from inputTbl1 ) a )b;
If the hive.merge.mapfiles and hive.merge.mapredfiles are turned off, I do not see any error.
Here is the stack trace:
2014-08-16 01:32:26,849 ERROR [main]: ql.Driver (SessionState.java:printError(681)) - FAILED: NullPointerException null java.lang.NullPointerException at org.apache.hadoop.hive.ql.optimizer.GenMapRedUtils.createMoveTask(GenMapRedUtils.java:1738) at org.apache.hadoop.hive.ql.parse.spark.GenSparkUtils.processFileSink(GenSparkUtils.java:281) at org.apache.hadoop.hive.ql.parse.spark.SparkCompiler.generateTaskTree(SparkCompiler.java:187) at org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:199) at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9508) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208) at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74) at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:208) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:414) at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:310) at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1005) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1070) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:942) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:932) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:246) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:198) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:781) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
Attachments
Attachments
Issue Links
- Is contained by
-
HIVE-7292 Hive on Spark
- Resolved