Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1900

Assign predicates below analytic functions with a compatible partition by clause.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.0, Impala 2.0.1, Impala 2.1.1
    • Impala 2.3.0
    • None

    Description

      We currently never push predicates below an analytic function because that would generally affect the analytic result. However, if the analytic function's partition by clause is "compatible" with the predicate to be pushed then we can and should allow the predicate to go below.
      For example, in the following query we could use the predicate on year for partition pruning:

      select * from
      (select year,
              sum(int_col) over(partition by year order by id)
       from functional.alltypes) v
      where year = 2009
      +--------------------------------------------------------------+
      | Explain String                                               |
      +--------------------------------------------------------------+
      | Estimated Per-Host Requirements: Memory=168.00MB VCores=2    |
      |                                                              |
      | 05:EXCHANGE [UNPARTITIONED]                                  |
      | |                                                            |
      | 03:SELECT                                                    |
      | |  predicates: year = 2009                                   |
      | |                                                            |
      | 02:ANALYTIC                                                  |
      | |  functions: sum(int_col)                                   |
      | |  partition by: year                                        |
      | |  order by: id ASC                                          |
      | |  window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW |
      | |                                                            |
      | 01:SORT                                                      |
      | |  order by: year ASC NULLS FIRST, id ASC                    |
      | |                                                            |
      | 04:EXCHANGE [HASH(year)]                                     |
      | |                                                            |
      | 00:SCAN HDFS [functional.alltypes]                           |
      |    partitions=24/24 files=24 size=478.45KB                   |
      +--------------------------------------------------------------+
      

      Attachments

        Activity

          People

            dtsirogiannis Dimitris Tsirogiannis
            alex.behm Alexander Behm
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: