[ARROW-9425] [Rust][DataFusion] Make ExecutionContext sharable between threads

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: Rust - DataFusion
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/25501
• Assignee: Jorge Leitão
• Reporter: Jorge Leitão
• Created: 13/Jul/20 07:32
• Updated: 11/Jan/23 08:06
• Resolved: 13/Jul/20 14:52

Description

I have been playing with pyo3 to ship and call this library directly from Python, and the current ExecutionContext can't be used within Python with threads due to the usage of `Box<dyn TableProvider>` on `datasources`, i.e. no `Sync` nor `Send`.

This issue likely affects integration of DataFusion with other libraries.

I propose that we allow ExecutionContext to be sharable between threads, e.g. using Box<dyn TableProvider + Send + Sync>.

Issue Links

• is related to: ARROW-9888 [Rust] [DataFusion] ExecutionContext can not be shared between threads
• links to: GitHub Pull Request #7723