[IMPALA-2328] Parquet scan should use min/max statistics to skip blocks based on predicate

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 2.2.4
• Fix Version/s: Impala 2.9.0
• Component/s: Backend
• Labels: observability performance
• Epic Link: Improve the reliability and effectiveness of ETL
• Target Version: Impala 2.9.0
• Assignee: Lars Volker
• Reporter: Mostafa Mokhtar
• Created: 12/Sep/15 00:44
• Updated: 22/Jun/17 22:36
• Resolved: 23/Feb/17 17:07

Description

Parquet stores min/max stats which can be used to skip reading blocks if they don't qualify a certain predicate

The query below ends up scanning all rows, which is not needed.

select count(*) from tpch_parquet.lineitem where l_orderkey = -1;

Issue Links

• relates to: IMPALA-3909 Parquet file writer should populate the min/max statistics per block per column to be used by the reader