Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1900

Assign predicates below analytic functions with a compatible partition by clause.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.0, Impala 2.0.1, Impala 2.1.1
    • Fix Version/s: Impala 2.3.0
    • Component/s: None
    • Labels:

      Description

      We currently never push predicates below an analytic function because that would generally affect the analytic result. However, if the analytic function's partition by clause is "compatible" with the predicate to be pushed then we can and should allow the predicate to go below.
      For example, in the following query we could use the predicate on year for partition pruning:

      select * from
      (select year,
              sum(int_col) over(partition by year order by id)
       from functional.alltypes) v
      where year = 2009
      +--------------------------------------------------------------+
      | Explain String                                               |
      +--------------------------------------------------------------+
      | Estimated Per-Host Requirements: Memory=168.00MB VCores=2    |
      |                                                              |
      | 05:EXCHANGE [UNPARTITIONED]                                  |
      | |                                                            |
      | 03:SELECT                                                    |
      | |  predicates: year = 2009                                   |
      | |                                                            |
      | 02:ANALYTIC                                                  |
      | |  functions: sum(int_col)                                   |
      | |  partition by: year                                        |
      | |  order by: id ASC                                          |
      | |  window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW |
      | |                                                            |
      | 01:SORT                                                      |
      | |  order by: year ASC NULLS FIRST, id ASC                    |
      | |                                                            |
      | 04:EXCHANGE [HASH(year)]                                     |
      | |                                                            |
      | 00:SCAN HDFS [functional.alltypes]                           |
      |    partitions=24/24 files=24 size=478.45KB                   |
      +--------------------------------------------------------------+
      

        Attachments

          Activity

            People

            • Assignee:
              dtsirogiannis Dimitris Tsirogiannis
              Reporter:
              alex.behm Alexander Behm
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: