[HIVE-24167] TPC-DS query 14 fails while generating plan for the filter

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: CBO
• Assignee: Zoltan Haindrich
• Reporter: Stamatis Zampetakis
• Created: 15/Sep/20 14:25
• Updated: 18/Nov/20 17:53

Description

TPC-DS query 14 (cbo_query14.q and query4.q) fail with NPE on the metastore with the partitioned TPC-DS 30TB dataset while generating the plan for the filter.

The problem can be reproduced using the PR in HIVE-23965.

The current stacktrace shows that the NPE appears while trying to display the debug message but even if this line didn't exist it would fail again later on.

java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10867)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11765)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11635)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlanForSubQueryPredicate(SemanticAnalyzer.java:3375)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genFilterPlan(SemanticAnalyzer.java:3473)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10819)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11765)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11625)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11625)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11622)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11649)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11635)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12417)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:718)
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12519)
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:443)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:301)
        at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:171)
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:301)
        at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
        at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:173)
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:414)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:363)
        at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:357)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:129)
        at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:231)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd1(CliDriver.java:203)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:129)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:424)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:355)
        at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:740)
        at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:710)
        at org.apache.hadoop.hive.cli.control.CorePerfCliDriver.runTest(CorePerfCliDriver.java:103)
        at org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
        at org.apache.hadoop.hive.cli.TestTezTPCDS30TBPerfCliDriver.testCliDriver(TestTezTPCDS30TBPerfCliDriver.java:83)

Issue Links

• relates to: HIVE-24233 except subquery throws nullpointer with cbo disabled