[HIVE-20955] Calcite Rule HiveExpandDistinctAggregatesRule seems throwing IndexOutOfBoundsException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Assignee: Vineet Garg
• Reporter: Slim Bouguerra
• Created: 21/Nov/18 15:44
• Updated: 10/Dec/18 22:03
• Resolved: 10/Dec/18 22:02

Description

 

Adde the following query to Druid test  ql/src/test/queries/clientpositive/druidmini_expressions.q

select count(distinct `__time`, cint) from (select * from druid_table_alltypesorc) as src;

leads to error {code} 2018-11-21T07:36:39,449 ERROR [main] QTestUtil: Client execution failed with error code = 40000 running "{code}

with exception stack 

2018-11-21T07:36:39,443 ERROR [ecd48683-0286-4cb4-b0ad-e150fab51038 main] parse.CalcitePlanner: CBO failed, skipping CBO.
java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
 at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:310) ~[guava-19.0.jar:?]
 at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:293) ~[guava-19.0.jar:?]
 at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:41) ~[guava-19.0.jar:?]
 at org.apache.calcite.rel.metadata.RelMdColumnOrigins.getColumnOrigins(RelMdColumnOrigins.java:77) ~[calcite-core-1.17.0.jar:1.17.0]
 at GeneratedMetadataHandler_ColumnOrigin.getColumnOrigins_$(Unknown Source) ~[?:?]
 at GeneratedMetadataHandler_ColumnOrigin.getColumnOrigins(Unknown Source) ~[?:?]
 at org.apache.calcite.rel.metadata.RelMetadataQuery.getColumnOrigins(RelMetadataQuery.java:345) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveExpandDistinctAggregatesRule.onMatch(HiveExpandDistinctAggregatesRule.java:168) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:315) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:556) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:415) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:280) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.plan.hep.HepInstruction$RuleCollection.execute(HepInstruction.java:74) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:211) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:198) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:2363) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:2314) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:2031) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1780) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1680) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:118) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1043) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:154) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:111) ~[calcite-core-1.17.0.jar:1.17.0]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1439) ~[hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:478) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12296) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:358) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:288) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:670) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1893) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1840) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1835) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:219) [hive-exec-4.0.0-SNAPSHOT.jar:4.0.0-SNAPSHOT]
 at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:242) [hive-cli-4.0.0-SNAPSHOT.jar:?]
 at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:189) [hive-cli-4.0.0-SNAPSHOT.jar:?]
 at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:408) [hive-cli-4.0.0-SNAPSHOT.jar:?]
 at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:341) [hive-cli-4.0.0-SNAPSHOT.jar:?]

Issue Links

• contains: HIVE-21026 Druid Vectorize Reader is not using the correct input size
• is blocked by: HIVE-21026 Druid Vectorize Reader is not using the correct input size