[ARROW-15732] [C++] Do not use any CPU threads in execution plan when use_threads is false

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 11.0.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31181
• Assignee: Unassigned
• Reporter: Weston Pace
• Created: 19/Feb/22 02:25
• Updated: 11/Jan/23 11:39
• Resolved: 30/Dec/22 18:56

Description

Currently, scan operations that supply use_threads=false will use 1 CPU thread, the calling thread, and some # of I/O threads. We should remove that CPU thread from being used.

It is being used at the moment because we need something to transfer work onto when asynchronous I/O tasks complete. We can fix this by allowing a serial executor to run an async generator (creating an iterator). This will also allow us to change the execution plan so that it always expects an executor to be present which should simplify the logic.

Issue Links

• blocks: ARROW-16179 [C++] Migrate existing usage of ThreadIndexer to ThreadLocalState and get rid of ThreadIndexer
• is related to: ARROW-17590 Lower memory usage with filters
• links to: GitHub Pull Request #12468
• links to: GitHub Pull Request #15104