Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4470

Predicate hits NumberFormatException due to constant folding.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:

      Description

      Impala folds constants in a few specific places. For example, for predicates assigned to Kudu scan nodes, or for predicates used as partition filters. This constant folding may result in queries not being executable if the evaluation result is NaN or infinity (and possibly other special values).

      The underlying problem is that our NumeritLiteral has no good way of representing NaN or infinity.

      Repro:

      explain select * from functional.alltypes
      where year < (cast(1.1000000238418579 as double) / cast(0 as double));
      ERROR: NumberFormatException: Infinite or NaN
      

      Stack:

      I1110 13:04:23.117324 16414 jni-util.cc:169] java.lang.NumberFormatException: Infinite or NaN
      	at java.math.BigDecimal.<init>(BigDecimal.java:808)
      	at org.apache.impala.analysis.LiteralExpr.create(LiteralExpr.java:202)
      	at org.apache.impala.analysis.Expr.foldConstantChildren(Expr.java:1219)
      	at org.apache.impala.planner.HdfsPartitionPruner.canEvalUsingPartitionMd(HdfsPartitionPruner.java:182)
      	at org.apache.impala.planner.HdfsPartitionPruner.prunePartitions(HdfsPartitionPruner.java:123)
      	at org.apache.impala.planner.SingleNodePlanner.createHdfsScanPlan(SingleNodePlanner.java:1199)
      	at org.apache.impala.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:1259)
      	at org.apache.impala.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:1482)
      	at org.apache.impala.planner.SingleNodePlanner.createTableRefsPlan(SingleNodePlanner.java:757)
      	at org.apache.impala.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:597)
      	at org.apache.impala.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:248)
      	at org.apache.impala.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:145)
      	at org.apache.impala.planner.Planner.createPlan(Planner.java:91)
      	at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:987)
      	at org.apache.impala.service.Frontend.createExecRequest(Frontend.java:1067)
      	at org.apache.impala.service.JniFrontend.createExecRequest(JniFrontend.java:155)
      

      The easiest solution is to consider the constant folding a failure for special evaluation results like NaN/infinity and fall back to executing the original expression.
      As a result, the index-based partition pruning may not kick in or predicates may not be pushed to Kudu. These cases seem exotic enough for this behavior to be acceptable in most cases.

      Alternatively, we could work on proper representation of NaN and infinity in our NumericLiteral.

        Attachments

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: