[ARROW-2801] [Python][C++][Dataset] Implement split_row_groups for ParquetDataset

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Component/s: Python
• Labels: dataset dataset-parquet-read parquet pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/19181
• Assignee: Unassigned
• Reporter: Robbie Gruener
• Created: 06/Jul/18 20:11
• Updated: 11/Jan/23 07:23
• Resolved: 15/Apr/21 08:26

Description

Currently the split_row_groups argument in ParquetDataset yields a not implemented error. An easy and efficient way to implement this is by using the summary metadata file instead of opening every footer file

Issue Links

• is blocked by: ARROW-1983 [Python] Add ability to write parquet `_metadata` file
• is related to: ARROW-8446 [Python][Dataset] Detect and use _metadata file in a list of file paths
• links to: GitHub Pull Request #2223
• links to: GitHub Pull Request #7515