[IMPALA-10405] Consider setting parquet_page_row_count_limit to 20000 by default

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Backend
• Labels: parquet performance
• Target Version: Product Backlog
• Epic Color: ghx-label-3
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 22/Dec/20 22:15
• Updated: 22/Dec/20 22:29

Description

PARQUET-1414 did some experiments for parquet-mr that concluded that this setting would enhance page filtering without giving up much compression. It's the default over there.

We should probably just do the same in Impala because we already have that evidence that it's better and we can avoid it being a confounding factor.

Issue Links

• relates to: IMPALA-8449 Avoid Parquet pages with too many rows + try to make them aligned
• relates to: IMPALA-7936 Enable better control over Parquet writing
• relates to: PARQUET-1414 Limit page size based on maximum row count