[ARROW-11616] [Rust][DataFusion] Expose collect_partitioned for DataFrame

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 4.0.0
• Component/s: Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/27481
• Assignee: Mike Seddon
• Reporter: Mike Seddon
• Created: 13/Feb/21 02:43
• Updated: 11/Jan/23 08:20
• Resolved: 16/Feb/21 04:54

Description

The DataFrame API has a `collect` method which invokes the `collect(plan: Arc<dyn ExecutionPlan>) -> Result<Vec<RecordBatch>>` function which will collect records into a single vector of RecordBatches removing the partitioning via `MergeExec`.

The DataFrame should also expose the `collect_partitioned` method so that partitions can be maintained.

```
collect_partitioned(
plan: Arc<dyn ExecutionPlan>,
) -> Result<Vec<Vec<RecordBatch>>>
```

Issue Links

• links to: GitHub Pull Request #9485