Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1047

ChunkList.clear throws AssertionError

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: core
    • Labels:
      None

      Description

      We are trying to enable VolcanoPlanner in Hive for some specific cases. Doing experiments, I tried to set up the impatient option to true. Then I am hitting an assertion error in ChunkList with the following stack trace:

      select cbo_t3.c_int, c, count(*) from (select key as a, c_int+1 as b, sum(c_int) as c from cbo_t1 where (cbo_t1.c_int + 1 >= 0) and (cbo_t1.c_int > 0 or cbo_t1.c_float >= 0) group by c_float, cbo_t1.c_int, key order by a) cbo_t1 join (select key as p, c_int+1 as q, sum(c_int) as r from cbo_t2 where (cbo_t2.c_int + 1 >= 0) and (cbo_t2.c_int > 0 or cbo_t2.c_float >= 0)  group by c_float, cbo_t2.c_int, key order by q/10 desc, r asc) cbo_t2 on cbo_t1.a=p join cbo_t3 on cbo_t1.a=key where (b + cbo_t2.q >= 0) and (b > 0 or c_int >= 0) group by cbo_t3.c_int, c order by cbo_t3.c_int+c desc, c
      See ./ql/target/tmp/log/hive.log or ./itests/qtest/target/tmp/log/hive.log, or check ./ql/target/surefire-reports or ./itests/qtest/target/surefire-reports/ for specific test cases logs.
        TestCliDriver.testCliDriver_correlationoptimizer8:108->runTest:134 Unexpected exception java.lang.AssertionError
      	at org.apache.calcite.util.ChunkList$ChunkListIterator.next(ChunkList.java:217)
      	at java.util.AbstractList.removeRange(AbstractList.java:570)
      	at java.util.AbstractList.clear(AbstractList.java:234)
      	at org.apache.calcite.plan.volcano.RuleQueue$PhaseMatchList.clear(RuleQueue.java:714)
      	at org.apache.calcite.plan.volcano.RuleQueue.phaseCompleted(RuleQueue.java:155)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:833)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.volcanoPlan(CalcitePlanner.java:1244)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.applyPreJoinOrderingTransforms(CalcitePlanner.java:1090)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:887)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:830)
      	at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:112)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:971)
      	at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:148)
      	at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:105)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:653)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:270)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:10094)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:229)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
      	at org.apache.hadoop.hive.ql.parse.ExplainSemanticAnalyzer.analyzeInternal(ExplainSemanticAnalyzer.java:74)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:237)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:471)
      	at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:311)
      	at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1149)
      	at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1237)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1075)
      	at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1063)
      	at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
      	at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:400)
      	at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:336)
      	at org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:1127)
      	at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:1101)
      	at org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
      	at org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_correlationoptimizer8(TestCliDriver.java:108)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      

      It seems the problem originates when we call the clean method in the ChunkList.

        Attachments

          Activity

            People

            • Assignee:
              julianhyde Julian Hyde
              Reporter:
              jcamachorodriguez Jesus Camacho Rodriguez
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: