[HIVE-13570] Some queries with Union all fail when CBO is off

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Assignee: Yongzhi Chen
• Reporter: Yongzhi Chen
• Created: 21/Apr/16 00:51
• Updated: 21/Jun/16 15:10
• Resolved: 24/Apr/16 23:59

Description

Some queries with union all throws IndexOutOfBoundsException
when:
set hive.cbo.enable=false;
set hive.ppd.remove.duplicatefilters=true;
The stack is as:

java.lang.IndexOutOfBoundsException: Index: 67, Size: 67 
        at java.util.ArrayList.rangeCheck(ArrayList.java:635) 
        at java.util.ArrayList.get(ArrayList.java:411) 
        at org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcCtx.genColLists(ColumnPrunerProcCtx.java:161) 
        at org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcCtx.handleFilterUnionChildren(ColumnPrunerProcCtx.java:273) 
        at org.apache.hadoop.hive.ql.optimizer.ColumnPrunerProcFactory$ColumnPrunerFilterProc.process(ColumnPrunerProcFactory.java:108) 
        at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90) 
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:94) 
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:78) 
        at org.apache.hadoop.hive.ql.optimizer.ColumnPruner$ColumnPrunerWalker.walk(ColumnPruner.java:172) 
        at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(DefaultGraphWalker.java:109) 
        at org.apache.hadoop.hive.ql.optimizer.ColumnPruner.transform(ColumnPruner.java:135) 
        at org.apache.hadoop.hive.ql.optimizer.Optimizer.optimize(Optimizer.java:198) 
        at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10327) 
        at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:192) 
        at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:222) 
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:432) 
        at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:305) 
        at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1119) 
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1167) 
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1055) 
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1045) 
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:207) 
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:159) 
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:370) 
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:305) 
        at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:403) 
        at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:419) 
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:708) 
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675) 
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:615)