Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-2029

Query with IS DISTINCT FROM condition in WHERE or JOIN clause fails with AssertionError: "Cast for just nullability not allowed"

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.15.0
    • Component/s: core
    • Labels:
      None

      Description

      Query

      select * from "foodmart"."employee" as e1 where e1."last_name" is distinct from e1."last_name"
      

      fails with error:

      java.lang.AssertionError: Cast for just nullability not allowed
      
      	at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
      	at org.apache.calcite.rel.core.Filter.isValid(Filter.java:116)
      	at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:353)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:101)
      	at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:336)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495)
      	at org.apache.calcite.plan.volcano.VolcanoPlanner.setRoot(VolcanoPlanner.java:308)
      	at org.apache.calcite.tools.Programs$5.run(Programs.java:309)
      	at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
      	at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:187)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:318)
      	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
      	at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
      	at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
      	at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
      	at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
      	at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
      	at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:564)
      	at org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1397)
      	at org.apache.calcite.test.JdbcTest.testMixedEqualAndIsNotDistinctJoin(JdbcTest.java:1593)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
      	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
      	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
      	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
      	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
      

      Method RelOptUtil.isDistinctFrom() makes an additional cast to NOT NULL BOOLEAN during the creation of the condition. This condition is not simplified before the time when it is passed into the filter constructor, therefore method Filter.isValid() throws AssertionError.

        Activity

        Hide
        julianhyde Julian Hyde added a comment -

        Resolved in release 1.15.0 (2017-12-11).

        Show
        julianhyde Julian Hyde added a comment - Resolved in release 1.15.0 (2017-12-11).
        Hide
        vvysotskyi Volodymyr Vysotskyi added a comment -

        Julian Hyde thanks for so quick response and reworking my pull request!

        Show
        vvysotskyi Volodymyr Vysotskyi added a comment - Julian Hyde thanks for so quick response and reworking my pull request!
        Show
        julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/61f1258c ; thanks for the PR, Volodymyr Vysotskyi !
        Hide
        julianhyde Julian Hyde added a comment -

        Your fix works, but I'd rather not make the LogicalFilter constructor do more than it does already. I think the real culprit is the code that converts x IS DISTINCT FROM y to CASE WHEN x IS NULL THEN y IS NOT NULL WHEN y IS NULL THEN x IS NOT NULL ELSE x <> y END. The expression never produces nulls in practice, so the CASE expression's type should reflect that. By the time of the ELSE x <> y we know that x and y are not null, so we should cast them before comparing them. I'm going to try making that change.

        Show
        julianhyde Julian Hyde added a comment - Your fix works, but I'd rather not make the LogicalFilter constructor do more than it does already. I think the real culprit is the code that converts x IS DISTINCT FROM y to CASE WHEN x IS NULL THEN y IS NOT NULL WHEN y IS NULL THEN x IS NOT NULL ELSE x <> y END . The expression never produces nulls in practice, so the CASE expression's type should reflect that. By the time of the ELSE x <> y we know that x and y are not null, so we should cast them before comparing them. I'm going to try making that change.
        Hide
        vvysotskyi Volodymyr Vysotskyi added a comment -

        I have created pull request for this Jira: https://github.com/apache/calcite/pull/554

        Show
        vvysotskyi Volodymyr Vysotskyi added a comment - I have created pull request for this Jira: https://github.com/apache/calcite/pull/554
        Hide
        vvysotskyi Volodymyr Vysotskyi added a comment -

        Query

        SELECT *
        FROM "foodmart"."employee" AS e1
        JOIN "foodmart"."employee" AS e2 ON e1."first_name" = e1."first_name"
        AND e1."last_name" IS DISTINCT
        FROM e2."last_name"
        

        fails with the same error, but stack trace is little bit different:

        java.lang.AssertionError: Cast for just nullability not allowed
        
        	at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31)
        	at org.apache.calcite.rel.core.Filter.isValid(Filter.java:116)
        	at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:353)
        	at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495)
        	at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863)
        	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883)
        	at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1766)
        	at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135)
        	at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234)
        	at org.apache.calcite.rel.convert.ConverterRule.onMatch(ConverterRule.java:117)
        	at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212)
        	at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650)
        	at org.apache.calcite.tools.Programs$5.run(Programs.java:326)
        	at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387)
        	at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:187)
        	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:318)
        	at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229)
        	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786)
        	at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640)
        	at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610)
        	at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221)
        	at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603)
        	at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638)
        	at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149)
        	at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218)
        	at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:564)
        	at org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1397)
        	at org.apache.calcite.test.JdbcTest.testMixedEqualAndIsNotDistinctJoin2(JdbcTest.java:1602)
        	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        	at java.lang.reflect.Method.invoke(Method.java:606)
        	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
        	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
        	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
        	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
        	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
        	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
        

        In the case of the query from Jira description, the filter is created by calling LogicalFilter.create(). In the case of the query from this comment, the filter is created during applying EnumerableJoinRule.convert() method.

        Show
        vvysotskyi Volodymyr Vysotskyi added a comment - Query SELECT * FROM "foodmart" . "employee" AS e1 JOIN "foodmart" . "employee" AS e2 ON e1. "first_name" = e1. "first_name" AND e1. "last_name" IS DISTINCT FROM e2. "last_name" fails with the same error, but stack trace is little bit different: java.lang.AssertionError: Cast for just nullability not allowed at org.apache.calcite.util.Litmus$1.fail(Litmus.java:31) at org.apache.calcite.rel.core.Filter.isValid(Filter.java:116) at org.apache.calcite.rel.AbstractRelNode.onRegister(AbstractRelNode.java:353) at org.apache.calcite.plan.volcano.VolcanoPlanner.registerImpl(VolcanoPlanner.java:1495) at org.apache.calcite.plan.volcano.VolcanoPlanner.register(VolcanoPlanner.java:863) at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:883) at org.apache.calcite.plan.volcano.VolcanoPlanner.ensureRegistered(VolcanoPlanner.java:1766) at org.apache.calcite.plan.volcano.VolcanoRuleCall.transformTo(VolcanoRuleCall.java:135) at org.apache.calcite.plan.RelOptRuleCall.transformTo(RelOptRuleCall.java:234) at org.apache.calcite.rel.convert.ConverterRule.onMatch(ConverterRule.java:117) at org.apache.calcite.plan.volcano.VolcanoRuleCall.onMatch(VolcanoRuleCall.java:212) at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:650) at org.apache.calcite.tools.Programs$5.run(Programs.java:326) at org.apache.calcite.tools.Programs$SequenceProgram.run(Programs.java:387) at org.apache.calcite.prepare.Prepare.optimize(Prepare.java:187) at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:318) at org.apache.calcite.prepare.Prepare.prepareSql(Prepare.java:229) at org.apache.calcite.prepare.CalcitePrepareImpl.prepare2_(CalcitePrepareImpl.java:786) at org.apache.calcite.prepare.CalcitePrepareImpl.prepare_(CalcitePrepareImpl.java:640) at org.apache.calcite.prepare.CalcitePrepareImpl.prepareSql(CalcitePrepareImpl.java:610) at org.apache.calcite.jdbc.CalciteConnectionImpl.parseQuery(CalciteConnectionImpl.java:221) at org.apache.calcite.jdbc.CalciteMetaImpl.prepareAndExecute(CalciteMetaImpl.java:603) at org.apache.calcite.avatica.AvaticaConnection.prepareAndExecuteInternal(AvaticaConnection.java:638) at org.apache.calcite.avatica.AvaticaStatement.executeInternal(AvaticaStatement.java:149) at org.apache.calcite.avatica.AvaticaStatement.executeQuery(AvaticaStatement.java:218) at org.apache.calcite.test.CalciteAssert.assertQuery(CalciteAssert.java:564) at org.apache.calcite.test.CalciteAssert$AssertQuery.runs(CalciteAssert.java:1397) at org.apache.calcite.test.JdbcTest.testMixedEqualAndIsNotDistinctJoin2(JdbcTest.java:1602) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.junit.runner.JUnitCore.run(JUnitCore.java:137) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) In the case of the query from Jira description, the filter is created by calling LogicalFilter.create() . In the case of the query from this comment, the filter is created during applying EnumerableJoinRule.convert() method.

          People

          • Assignee:
            julianhyde Julian Hyde
            Reporter:
            vvysotskyi Volodymyr Vysotskyi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development