[IMPALA-10325] Parquet scan should use min/max statistics to skip pages based on equi-join predicate

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: Impala 4.0.0
• Component/s: Backend
• Epic Color: ghx-label-11
• Assignee: Qifan Chen
• Reporter: Qifan Chen
• Created: 13/Nov/20 14:43
• Updated: 02/Nov/23 00:03
• Resolved: 23/Feb/21 17:50

Description

Parquet stores min/max stats for pages which can be used to skip certain pages if they don't qualify an equi-join predicate.

The query below ends up scanning all rows for table a, which may not be needed if the min/max of b.ss_addr_sk can be detected and used during the scan of a.

select a.ss_sold_time_sk from
store_sales a join [SHUFFLE] store_sales b
where a.ss_addr_sk = b.ss_addr_sk and
b.ss_customer_sk < 10
;

Issue Links

• relates to: IMPALA-3430 Runtime filter : Extend runtime filter to support Min/Max values for HDFS scans