[ARROW-14464] [R] Change write_parquet()'s default chunk_size from all rows

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.0.0
• Component/s: R
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/30025
• Assignee: Jonathan Keane
• Reporter: Jonathan Keane
• Created: 25/Oct/21 20:25
• Updated: 11/Jan/23 08:40
• Resolved: 10/Nov/21 14:00

Description

Currently, when chunk_size is not given in write_parquet() it defaults to 1 chunk of all of the rows in the table. This could be fine for smallish numbers of rows, but when we have lots of rows, we want files that contain a decent number of row groups.

It looks like this was added in https://github.com/apache/arrow/pull/5451 and wasn't discussed there, so this default might not be intentional.

Issue Links

• links to: GitHub Pull Request #11613