[ARROW-7800] [Python] Expose GetRecordBatchReader API in PyArrow

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Python
• Labels: features pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/24033
• Assignee: Will Jones
• Reporter: Will Jones
• Created: 07/Feb/20 20:38
• Updated: 11/Jan/23 07:56
• Resolved: 06/Jan/21 10:16

Description

The GetRecordBatchReader API is really useful for streaming ParquetFiles with lots of RLE.

I propose exposing this API in PyArrow in the following manner:

file_ = ParquetFile('file/path.parquet')

for batch in file_.get_batches(batch_size=100):
     pass

(If anyone has any better ideas hit me up, I'm not 100% sold on exposing it this way.)

Issue Links

• links to: GitHub Pull Request #6386
• links to: GitHub Pull Request #6979