[ARROW-13542] [C++][Compute][Dataset] Add dataset::WriteNode for writing rows from an ExecPlan to disk

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 6.0.0
• Component/s: C++
• Labels: dataset pull-request-available query-engine
• External issue URL: https://github.com/apache/arrow/issues/29194
• Assignee: Weston Pace
• Reporter: Ben Kietzman
• Created: 03/Aug/21 20:23
• Updated: 11/Jan/23 08:33
• Resolved: 08/Oct/21 00:54

Description

This will serve as a sink ExecNode which dumps all the batches it receives to disk. The PR should probably also replace FileSystemDataset::Write with an ExecPlan based implementation

Issue Links

• is a child of: ARROW-12633 [C++] Query engine umbrella issue
• is depended upon by: ARROW-14191 [C++][Dataset] Dataset writes should respect backpressure
• is depended upon by: ARROW-13554 [C++] Remove deprecated Scanner::Scan
• links to: GitHub Pull Request #11017

Sub-Tasks

• [C++] Create LruCache that works with futures: Closed
• [C++] Create dataset writer to encapsulate dataset writer logic: Resolved