Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1860

Duplicate null predicates cause NullPointerException in RexUtil

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.14.0
    • Component/s: None
    • Labels:
      None
    1. screenshot-1.png
      483 kB
      Ruidong Li
    2. screenshot-2.png
      165 kB
      Ruidong Li

      Activity

      Hide
      michaelmior Michael Mior added a comment -

      Resolved in release 1.14.0 (2017-10-01)

      Show
      michaelmior Michael Mior added a comment - Resolved in release 1.14.0 (2017-10-01)
      Hide
      RuidongLi Ruidong Li added a comment - - edited

      Thanks for your advice, I'll change my github account name to my real name.

      Show
      RuidongLi Ruidong Li added a comment - - edited Thanks for your advice, I'll change my github account name to my real name.
      Hide
      julianhyde Julian Hyde added a comment -
      Show
      julianhyde Julian Hyde added a comment - Fixed in http://git-wip-us.apache.org/repos/asf/calcite/commit/5a91d0a4 . Thanks for the PR, Ruidong Li !
      Hide
      julianhyde Julian Hyde added a comment -

      Could you perhaps change the name attached to your github account? A code contribution is a legal transaction. And Apache committers are the gate-keepers. It makes it easier for us if your real name is attached to the commit in github.

      Show
      julianhyde Julian Hyde added a comment - Could you perhaps change the name attached to your github account? A code contribution is a legal transaction. And Apache committers are the gate-keepers. It makes it easier for us if your real name is attached to the commit in github.
      Show
      julianhyde Julian Hyde added a comment - See https://github.com/julianhyde/calcite/commit/352f573e83e7d3f6c7d187beedaa39e8a188dea7 . Not pushed to master yet.
      Hide
      RuidongLi Ruidong Li added a comment -

      can you also show your test case which does not involve UDFs?

      Show
      RuidongLi Ruidong Li added a comment - can you also show your test case which does not involve UDFs?
      Hide
      RuidongLi Ruidong Li added a comment -

      Yes, My github account name is Xpray

      Show
      RuidongLi Ruidong Li added a comment - Yes, My github account name is Xpray
      Hide
      julianhyde Julian Hyde added a comment -

      Looks good. I have managed to create a test case that does not involve UDFs,

      Are you Xpray? I would like to use your real name in the git commit message.

      Show
      julianhyde Julian Hyde added a comment - Looks good. I have managed to create a test case that does not involve UDFs, Are you Xpray? I would like to use your real name in the git commit message.
      Hide
      RuidongLi Ruidong Li added a comment - - edited

      This sql query will cause this problem

      select a, b, c from 
      (select a, b, c, createNull() as d from inputTable) 
      where d is null
      

      CreateNull is a UDF with isDeterministic = true and always returns null

      Show
      RuidongLi Ruidong Li added a comment - - edited This sql query will cause this problem select a, b, c from (select a, b, c, createNull() as d from inputTable) where d is null CreateNull is a UDF with isDeterministic = true and always returns null
      Hide
      RuidongLi Ruidong Li added a comment - - edited

      and the Flink-TableAPI program:

      I think it can't be replayed by using SQL, the Flink-TableAPI UDF's `isDeterministic` returns `true` by default,so the `ReduceExpressionsRule` will treat `CreateNull()` as a pure function, when I set isDeterministic to false, no error happends

      Timo Walther please hava a look at this

      Show
      RuidongLi Ruidong Li added a comment - - edited and the Flink-TableAPI program: I think it can't be replayed by using SQL, the Flink-TableAPI UDF's `isDeterministic` returns `true` by default,so the `ReduceExpressionsRule` will treat `CreateNull()` as a pure function, when I set isDeterministic to false, no error happends Timo Walther please hava a look at this
      Hide
      RuidongLi Ruidong Li added a comment - - edited

      the stack when NPE happens

      Exception in thread "main" java.lang.NullPointerException
      	at org.apache.calcite.rex.RexUtil.gatherConstraint(RexUtil.java:435)
      	at org.apache.calcite.rex.RexUtil.gatherConstraints(RexUtil.java:397)
      	at org.apache.calcite.rex.RexUtil.predicateConstants(RexUtil.java:334)
      	at org.apache.calcite.plan.RelOptPredicateList.of(RelOptPredicateList.java:144)
      	at org.apache.calcite.plan.RelOptPredicateList.union(RelOptPredicateList.java:157)
      	at org.apache.calcite.rel.metadata.RelMdPredicates.getPredicates(RelMdPredicates.java:287)
      	at GeneratedMetadataHandler_Predicates.getPredicates_$(Unknown Source)
      	at GeneratedMetadataHandler_Predicates.getPredicates(Unknown Source)
      	at GeneratedMetadataHandler_Predicates.getPredicates_$(Unknown Source)
      	at GeneratedMetadataHandler_Predicates.getPredicates(Unknown Source)
      	at org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:743)
      	at org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:267)
      	at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
      	at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:506)
      	at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:385)
      	at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251)
      	at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:125)
      	at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210)
      	at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:197)
      	at org.apache.flink.table.api.TableEnvironment.runHepPlanner(TableEnvironment.scala:255)
      	at org.apache.flink.table.api.StreamTableEnvironment.optimize(StreamTableEnvironment.scala:610)
      	at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:671)
      	at org.apache.flink.table.api.scala.StreamTableEnvironment.toDataStream(StreamTableEnvironment.scala:147)
      	at org.apache.flink.table.api.scala.TableConversions.toDataStream(TableConversions.scala:52)
      	at test.scala.WhereTest$delayedInit$body.apply(WhereTest.scala:34)
      	at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
      	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
      	at scala.App$$anonfun$main$1.apply(App.scala:71)
      	at scala.App$$anonfun$main$1.apply(App.scala:71)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
      	at scala.App$class.main(App.scala:71)
      	at test.scala.WhereTest$.main(WhereTest.scala:22)
      	at test.scala.WhereTest.main(WhereTest.scala)
      
      Show
      RuidongLi Ruidong Li added a comment - - edited the stack when NPE happens Exception in thread "main" java.lang.NullPointerException at org.apache.calcite.rex.RexUtil.gatherConstraint(RexUtil.java:435) at org.apache.calcite.rex.RexUtil.gatherConstraints(RexUtil.java:397) at org.apache.calcite.rex.RexUtil.predicateConstants(RexUtil.java:334) at org.apache.calcite.plan.RelOptPredicateList.of(RelOptPredicateList.java:144) at org.apache.calcite.plan.RelOptPredicateList.union(RelOptPredicateList.java:157) at org.apache.calcite.rel.metadata.RelMdPredicates.getPredicates(RelMdPredicates.java:287) at GeneratedMetadataHandler_Predicates.getPredicates_$(Unknown Source) at GeneratedMetadataHandler_Predicates.getPredicates(Unknown Source) at GeneratedMetadataHandler_Predicates.getPredicates_$(Unknown Source) at GeneratedMetadataHandler_Predicates.getPredicates(Unknown Source) at org.apache.calcite.rel.metadata.RelMetadataQuery.getPulledUpPredicates(RelMetadataQuery.java:743) at org.apache.calcite.rel.rules.ReduceExpressionsRule$ProjectReduceExpressionsRule.onMatch(ReduceExpressionsRule.java:267) at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317) at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:506) at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:385) at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251) at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:125) at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210) at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:197) at org.apache.flink.table.api.TableEnvironment.runHepPlanner(TableEnvironment.scala:255) at org.apache.flink.table.api.StreamTableEnvironment.optimize(StreamTableEnvironment.scala:610) at org.apache.flink.table.api.StreamTableEnvironment.translate(StreamTableEnvironment.scala:671) at org.apache.flink.table.api.scala.StreamTableEnvironment.toDataStream(StreamTableEnvironment.scala:147) at org.apache.flink.table.api.scala.TableConversions.toDataStream(TableConversions.scala:52) at test.scala.WhereTest$delayedInit$body.apply(WhereTest.scala:34) at scala.Function0$class.apply$mcV$sp(Function0.scala:40) at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12) at scala.App$$anonfun$main$1.apply(App.scala:71) at scala.App$$anonfun$main$1.apply(App.scala:71) at scala.collection.immutable.List.foreach(List.scala:318) at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32) at scala.App$class.main(App.scala:71) at test.scala.WhereTest$.main(WhereTest.scala:22) at test.scala.WhereTest.main(WhereTest.scala)
      Hide
      julianhyde Julian Hyde added a comment -

      Can you put the call stack in this JIRA case, in text so that people can find it via google.

      Also can you please provide a test case? A SQL query ideally, but a method in RexProgramTest would also work.

      Show
      julianhyde Julian Hyde added a comment - Can you put the call stack in this JIRA case, in text so that people can find it via google. Also can you please provide a test case? A SQL query ideally, but a method in RexProgramTest would also work.
      Hide
      RuidongLi Ruidong Li added a comment -

      I hava fix this and submit a PR at https://github.com/apache/calcite/pull/484

      Show
      RuidongLi Ruidong Li added a comment - I hava fix this and submit a PR at https://github.com/apache/calcite/pull/484

        People

        • Assignee:
          julianhyde Julian Hyde
          Reporter:
          RuidongLi Ruidong Li
        • Votes:
          0 Vote for this issue
          Watchers:
          3 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development