Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4213

Planner not pushing some predicates with constant exprs to Kudu

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.8.0
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:

      Description

      The following query is unable to push to Kudu:

      select count(*) from metrics where `timestamp` < 1475059765 + 10
      

      Oddly, the following one works fine:

      select count(*) from metrics where `timestamp` < 1000 + 10
      

      I'm guessing that some kind of implicit CAST is getting inserted here which is blocking the pushdown. 'timestamp' is an 'int' column here.

        Activity

        Hide
        mjacobs Matthew Jacobs added a comment -

        commit 2b5d1344c9a0a86807a54c7bdaa643c531a52054
        Author: Matthew Jacobs <mj@cloudera.com>
        Date: Mon Oct 3 17:00:48 2016 -0700

        IMPALA-4213: Fix Kudu predicates that need constant folding

        Folding const exprs where there were implicit casts on the
        slot resulted in the predicate not being pushed to Kudu.

        Change-Id: I3bab22d90ee00a054c847de6c734b4f24a3f5a85
        Reviewed-on: http://gerrit.cloudera.org:8080/4613
        Reviewed-by: Matthew Jacobs <mj@cloudera.com>
        Tested-by: Internal Jenkins

        Show
        mjacobs Matthew Jacobs added a comment - commit 2b5d1344c9a0a86807a54c7bdaa643c531a52054 Author: Matthew Jacobs <mj@cloudera.com> Date: Mon Oct 3 17:00:48 2016 -0700 IMPALA-4213 : Fix Kudu predicates that need constant folding Folding const exprs where there were implicit casts on the slot resulted in the predicate not being pushed to Kudu. Change-Id: I3bab22d90ee00a054c847de6c734b4f24a3f5a85 Reviewed-on: http://gerrit.cloudera.org:8080/4613 Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Internal Jenkins
        Hide
        alex.behm Alexander Behm added a comment -

        Actually it looks like we should be folding Kudu predicates, so it looks like a bug in the BinaryPredicate.normalizeSlotRefComparison() code

        Show
        alex.behm Alexander Behm added a comment - Actually it looks like we should be folding Kudu predicates, so it looks like a bug in the BinaryPredicate.normalizeSlotRefComparison() code
        Hide
        alex.behm Alexander Behm added a comment -

        The problem is our lack of constant folding.
        1475059765 + 10 --> INT + TINYINT = BIGINT

        So the timestamp INT col gets implicitly cast to BIGINT to match the right-hand side type.

        Show
        alex.behm Alexander Behm added a comment - The problem is our lack of constant folding. 1475059765 + 10 --> INT + TINYINT = BIGINT So the timestamp INT col gets implicitly cast to BIGINT to match the right-hand side type.

          People

          • Assignee:
            mjacobs Matthew Jacobs
            Reporter:
            tlipcon Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development