Uploaded image for project: 'Calcite'
  1. Calcite
  2. CALCITE-1738

Support CAST of literal values in filters pushed to Druid

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.13.0
    • Component/s: druid
    • Labels:
      None

      Description

      Because HIVE-16027 forced an implicit CAST on queries like WHERE <timestampcolumn> IN ('<literal', '<literal>'), IN, BETEWEEN and other filters are no longer pushed down to Druid. In this call stack:

      org.apache.calcite.adapter.druid.DruidDateTimeUtils.literalValue(DruidDateTimeUtils.java:246)
      	at org.apache.calcite.adapter.druid.DruidDateTimeUtils.leafToRanges(DruidDateTimeUtils.java:227)
      	at org.apache.calcite.adapter.druid.DruidDateTimeUtils.extractRanges(DruidDateTimeUtils.java:120)
      	at org.apache.calcite.adapter.druid.DruidDateTimeUtils.createInterval(DruidDateTimeUtils.java:65)
      	at org.apache.calcite.adapter.druid.DruidRules$DruidFilterRule.onMatch(DruidRules.java:186)
      	at org.apache.calcite.plan.AbstractRelOptPlanner.fireRule(AbstractRelOptPlanner.java:317)
      	at org.apache.calcite.plan.hep.HepPlanner.applyRule(HepPlanner.java:506)
      	at org.apache.calcite.plan.hep.HepPlanner.applyRules(HepPlanner.java:385)
      	at org.apache.calcite.plan.hep.HepPlanner.executeInstruction(HepPlanner.java:251)
      	at org.apache.calcite.plan.hep.HepInstruction$RuleInstance.execute(HepInstruction.java:125)
      	at org.apache.calcite.plan.hep.HepPlanner.executeProgram(HepPlanner.java:210)
      	at org.apache.calcite.plan.hep.HepPlanner.findBestExp(HepPlanner.java:197)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.hepPlan(CalcitePlanner.java:1790)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1518)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1265)
      	at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113)
      	at org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:1043)
      	at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149)
      	at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1073)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.getOptimizedAST(CalcitePlanner.java:1089)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:368)
      	at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:11119)
      	at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:290)
      	at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
      

      the literalValue only knows how to handle RexLiteral case. Because of the CAST, the node is a RexCall.

        Attachments

          Activity

            People

            • Assignee:
              rusanu Remus Rusanu
              Reporter:
              rusanu Remus Rusanu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: