[IMPALA-1900] Assign predicates below analytic functions with a compatible partition by clause.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.0, Impala 2.0.1, Impala 2.1.1
• Fix Version/s: Impala 2.3.0
• Labels: performance
• Assignee: Dimitris Tsirogiannis
• Reporter: Alexander Behm
• Created: 18/Mar/15 07:10
• Updated: 24/Aug/16 17:07
• Resolved: 07/Apr/15 01:49

Description

We currently never push predicates below an analytic function because that would generally affect the analytic result. However, if the analytic function's partition by clause is "compatible" with the predicate to be pushed then we can and should allow the predicate to go below.
For example, in the following query we could use the predicate on year for partition pruning:

select * from
(select year,
        sum(int_col) over(partition by year order by id)
 from functional.alltypes) v
where year = 2009
+--------------------------------------------------------------+
| Explain String                                               |
+--------------------------------------------------------------+
| Estimated Per-Host Requirements: Memory=168.00MB VCores=2    |
|                                                              |
| 05:EXCHANGE [UNPARTITIONED]                                  |
| |                                                            |
| 03:SELECT                                                    |
| |  predicates: year = 2009                                   |
| |                                                            |
| 02:ANALYTIC                                                  |
| |  functions: sum(int_col)                                   |
| |  partition by: year                                        |
| |  order by: id ASC                                          |
| |  window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW |
| |                                                            |
| 01:SORT                                                      |
| |  order by: year ASC NULLS FIRST, id ASC                    |
| |                                                            |
| 04:EXCHANGE [HASH(year)]                                     |
| |                                                            |
| 00:SCAN HDFS [functional.alltypes]                           |
|    partitions=24/24 files=24 size=478.45KB                   |
+--------------------------------------------------------------+