[ARROW-11791] [Rust][DataFusion] RepartitionExec Blocking

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.1, 4.0.0
• Component/s: Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/27642
• Assignee: Mike Seddon
• Reporter: Mike Seddon
• Created: 26/Feb/21 00:29
• Updated: 11/Jan/23 08:22
• Resolved: 26/Feb/21 21:44

Description

After https://github.com/apache/arrow/pull/9523 RepartitionExec is pulling all data into memory before starting the stream which crashes on large sets.

Issue Links

• links to: GitHub Pull Request #9580