[ARROW-357] Default Parquet chunk_size of 64k is too small

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.1.0
• Fix Version/s: 0.2.0
• Component/s: Python
• External issue URL: https://github.com/apache/arrow/issues/15329
• Assignee: Uwe Korn
• Reporter: Uwe Korn
• Created: 01/Nov/16 12:39
• Updated: 11/Jan/23 07:08
• Resolved: 02/Nov/16 16:20

Description

The current default size of RowGroups when writing Parquet files is 64k rows. This produces a lot of RowGroups given large files which are then really ineffective to read.