[ARROW-4603] [Rust] [DataFusion] Execution context should allow in-memory data sources to be registered

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: Rust, Rust - DataFusion
• External issue URL: https://github.com/apache/arrow/issues/21143
• Assignee: Andy Grove
• Reporter: Andy Grove
• Created: 17/Feb/19 16:04
• Updated: 11/Jan/23 07:35
• Resolved: 26/Feb/19 05:38

Description

Currently it is only possible to run SQL queries against CSV files. I would like to be able to run SQL queries against data that is already loaded into memory in a single RecordBatch or an iterator over RecordBatch.

There are two reasons I want this feature:

1. Ability to call DataFusion via IPC or JNI from Java to run queries
2. Ability to create benchmarks for the query engine that do not involve disk I/O

 

Issue Links

• is duplicated by: ARROW-4679 [Rust] [DataFusion] Implement in-memory DataSource