Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1479

AssertionError in ReduceExpressionsRule on multi-column IN sub-query

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.11.0
    • Component/s: core
    • Labels:

      Description

      See also https://github.com/apache/calcite/pull/317. This seems to be related to visiting subqueries without accounting for the fact that they might push multiple constancys on the stack.

      The assertion error I'm getting in my code is from the sql,

      SELECT * FROM s.foo
       WHERE (dim1, dim2) IN (
         SELECT dim1, dim2 FROM (
           SELECT dim1, dim2
           FROM s.foo
           GROUP BY dim1, dim2
         )
       )
      

      and the stack trace is,

      java.lang.AssertionError: Internal error: Error while applying rule ReduceExpressionsRule(Filter), args [rel#8:LogicalFilter.NONE.[](input=rel#7:Subset#0.NONE.[],condition=IN($2, $3, {
      LogicalProject(DIM1=[$0], DIM2=[$1])
        LogicalAggregate(group=[{0, 1}])
          LogicalProject(DIM1=[$2], DIM2=[$3])
            DruidQueryRel(dataSource=[foo])
      }))]
      
      	at org.apache.calcite.util.Util.newInternal(Util.java:792)
      	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:236)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:819)
      	at org.apache.calcite.tools.Programs$5.run(Programs.java:300)
      	at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:351)
      	at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:153)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:287)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:193)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:734)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:597)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:567)
      	at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:215)
      	at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:594)
      	at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:613)
      	at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:139)
      	at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:208)
      	at io.druid.sql.calcite.table.DruidTableTest.testQuery(DruidTableTest.java:1800)
      	at io.druid.sql.calcite.table.DruidTableTest.testUsingSubqueryAsFilterOnTwoColumns(DruidTableTest.java:1715)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:117)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:42)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:262)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
      Caused by: java.lang.AssertionError
      	at org.apache.calcite.rel.rules.ReduceExpressionsRule$ReducibleExprLocator.analyze(ReduceExpressionsRule.java:924)
      	at org.apache.calcite.rel.rules.ReduceExpressionsRule.findReducibleExps(ReduceExpressionsRule.java:603)
      	at org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressionsInternal(ReduceExpressionsRule.java:492)
      	at org.apache.calcite.rel.rules.ReduceExpressionsRule.reduceExpressions(ReduceExpressionsRule.java:462)
      	at org.apache.calcite.rel.rules.ReduceExpressionsRule$FilterReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:149)
      	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:213)
      	... 45 more
      

        Activity

        Hide
        julianhyde Julian Hyde added a comment -

        Gian Merlino, The problem was subtle: you were missing a call to .expand(false) in RelOptRulesTest, and as a result the sql-to-rel converter was removing RexSubQuery before you even saw them. I have a test case now, and I'll commit shortly.

        Show
        julianhyde Julian Hyde added a comment - Gian Merlino , The problem was subtle: you were missing a call to .expand(false) in RelOptRulesTest , and as a result the sql-to-rel converter was removing RexSubQuery before you even saw them. I have a test case now, and I'll commit shortly.
        Hide
        julianhyde Julian Hyde added a comment -
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/03d6b00c ; thanks for the PR, Gian Merlino !
        Hide
        gian Gian Merlino added a comment -

        Ah okay, thanks for the explanation.

        Show
        gian Gian Merlino added a comment - Ah okay, thanks for the explanation.
        Hide
        julianhyde Julian Hyde added a comment -

        Resolved in release 1.11.0 (2017-01-11).

        Show
        julianhyde Julian Hyde added a comment - Resolved in release 1.11.0 (2017-01-11).

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            gian Gian Merlino
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development