[IMPALA-5843] Use page index in Parquet files to skip pages

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0
• Fix Version/s: Impala 3.3.0
• Component/s: Backend
• Labels: parquet performance
• Epic Link: Improve the reliability and effectiveness of ETL
• Target Version: Product Backlog
• Epic Color: ghx-label-7
• Assignee: Zoltán Borók-Nagy
• Reporter: Lars Volker
• Created: 24/Aug/17 20:36
• Updated: 21/May/19 21:05
• Resolved: 13/May/19 11:56

Description

Once IMPALA-5842 has been resolved, we should skip pages based on the page index in Parquet files.

Issue Links

• is related to: IMPALA-6673 Be smarter about I/O patterns for Parquet scan ranges
• is related to: IMPALA-8449 Avoid Parquet pages with too many rows + try to make them aligned
• is related to: IMPALA-8441 Extend test_scanners_fuzz.py with selective queries
• relates to: IMPALA-3430 Runtime filter : Extend runtime filter to support Min/Max values for HDFS scans
• relates to: IMPALA-4625 Parquet Row Group Size optimization
• requires: IMPALA-5842 Write page index in Parquet files

Sub-Tasks

• Enhance scan ranges to support sub-ranges: Resolved
• Enhance decoders to support value-skipping: Resolved
• Impala 3.3 Doc: Doc Parquet page index: Closed