Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2119

Druid Filter validation Logic broken for filters like column_A = column_B

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.15.0
    • Fix Version/s: 1.16.0
    • Component/s: druid-adapter
    • Labels:
      None

      Description

      Currently, the logic for Filter tree validation and Filter translation to Druid native JSON is in a two different functions.
      Ideal to avoid this kind of runtime exceptions, we can blend both path of +Filter push down validation function +org.apache.calcite.adapter.druid.DruidQuery#isValidFilter(org.apache.calcite.rex.RexNode)
      and the +Translation function +org.apache.calcite.adapter.druid.DruidQuery.Translator#translateFilter.
      IMO, an easy implementation will be to try generating Druid native filter treat exceptions or null instance as it can not be pushed down. This will make code more readable and less duplication of logic that leads to fewer runtime exceptions.
      The following test

       @Test
        public void testFilterColumnAEqColumnB() {
          final String sql = "SELECT count(*) from \"foodmart\" where \"product_id\" = \"city\"";
          sql(sql, FOODMART).runs();
        }
       

      retruns

       
      
      java.lang.AssertionError: it is not a valid comparison: =($1, $29)
      
      	at org.apache.calcite.adapter.druid.DruidQuery$Translator.translateFilter(DruidQuery.java:1234)
      	at org.apache.calcite.adapter.druid.DruidQuery$Translator.access$000(DruidQuery.java:1114)
      	at org.apache.calcite.adapter.druid.DruidQuery.getQuery(DruidQuery.java:525)
      	at org.apache.calcite.adapter.druid.DruidQuery.deriveQuerySpec(DruidQuery.java:495)
      	at org.apache.calcite.adapter.druid.DruidQuery.getQuerySpec(DruidQuery.java:434)
      	at org.apache.calcite.adapter.druid.DruidQuery.deriveRowType(DruidQuery.java:324)
      	at org.apache.calcite.rel.AbstractRelNode.getRowType(AbstractRelNode.java:224)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:857)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1766)
      	at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
      	at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
      	at org.apache.calcite.adapter.druid.DruidRules$DruidFilterRule.onMatch(DruidRules.java:283)
      	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
      	at org.apache.calcite.tools.Programs$5.run(Programs.java:326)
      	at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
      	at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:188)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:319)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:230)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:781)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
      	at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
      	at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
      	at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
      	at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
      	at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
      	at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:568)
      	at org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1407)
      	at org.apache.calcite.test.DruidAdapterIT.testFilterColumnAEqColumnB(DruidAdapterIT.java:3494)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
      	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bslim Slim Bouguerra
                Reporter:
                bslim Slim Bouguerra
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: