[ARROW-7787] [Rust] Add collect to Table API

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.17.0
• Component/s: Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/24021
• Assignee: Jorge
• Reporter: Jorge Leitão
• Created: 06/Feb/20 20:27
• Updated: 11/Jan/23 07:56
• Resolved: 11/Feb/20 01:21

Description

Currently, executing using the table API requires some effort: given a table `t`:

plan = t.to_logical_plan()
plan = ctx.optimize(plan)
plan = ctx.create_physical_plan(plan, batch_size)
result = ctx.collect(plan)

This issue proposes 2 new public methods, one for Table,

fn collect(&self, ctx: &mut ExecutionContext, batch_size: usize) -> Result<Vec<RecordBatch>>;

and one for ExecutionContext,

pub fn collect_plan(&mut self, plan: &LogicalPlan, batch_size: usize) -> Result<Vec<RecordBatch>>

that optimize, execute and collect the results of the Table/LogicalPlan respectively, in the same spirit of `ExecutionContext.sql`.

 

 

 

Issue Links

• links to: GitHub Pull Request #6375