[PARQUET-2176] Parquet writers should allow for configurable index/statistics truncation

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.12.3
• Fix Version/s: 1.13.0
• Component/s: parquet-mr
• Assignee: Unassigned
• Reporter: patchwork01
• Created: 24/Aug/22 11:17
• Updated: 12/Apr/23 01:30
• Resolved: 26/Mar/23 06:06

Description

ParquetWriter does not expose any way to set the properties for column index or statistics truncation.

With ParquetOutputFormat those can be set with parquet.columnindex.truncate.length and parquet.statistics.truncate.length. These are not applied for ParquetWriter.

These properties are documented here: https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/README.md