[IMPALA-5842] Write page index in Parquet files

• Type: New Feature
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0
• Fix Version/s: Impala 2.13.0, Impala 3.1.0
• Component/s: Backend
• Labels: parquet
• Epic Link: Improve the reliability and effectiveness of ETL
• Target Version: Product Backlog
• Epic Color: ghx-label-6
• Assignee: Zoltán Borók-Nagy
• Reporter: Lars Volker
• Created: 24/Aug/17 20:34
• Updated: 28/Feb/19 00:10
• Resolved: 24/May/18 09:04

Description

Once PARQUET-922 has been resolved, we should start writing page indices to Parquet files.

Issue Links

• breaks: IMPALA-8257 Parquet writer sometimes hits DCHECK when handling empty string
• is duplicated by: IMPALA-5841 Write page index in Parquet files
• is related to: IMPALA-5840 Don't write page level statistics in Parquet files in anticipation of page indexes
• is required by: IMPALA-5843 Use page index in Parquet files to skip pages
• requires: PARQUET-922 Add index pages to the format to support efficient page skipping