Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.0, Impala 2.0.1, Impala 2.1.1
-
None
Description
We currently never push predicates below an analytic function because that would generally affect the analytic result. However, if the analytic function's partition by clause is "compatible" with the predicate to be pushed then we can and should allow the predicate to go below.
For example, in the following query we could use the predicate on year for partition pruning:
select * from
(select year,
sum(int_col) over(partition by year order by id)
from functional.alltypes) v
where year = 2009
+--------------------------------------------------------------+
| Explain String |
+--------------------------------------------------------------+
| Estimated Per-Host Requirements: Memory=168.00MB VCores=2 |
| |
| 05:EXCHANGE [UNPARTITIONED] |
| | |
| 03:SELECT |
| | predicates: year = 2009 |
| | |
| 02:ANALYTIC |
| | functions: sum(int_col) |
| | partition by: year |
| | order by: id ASC |
| | window: RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW |
| | |
| 01:SORT |
| | order by: year ASC NULLS FIRST, id ASC |
| | |
| 04:EXCHANGE [HASH(year)] |
| | |
| 00:SCAN HDFS [functional.alltypes] |
| partitions=24/24 files=24 size=478.45KB |
+--------------------------------------------------------------+