[ARROW-14266] [R] Use WriteNode to write queries

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: 8.0.0
• Component/s: R
• Labels: query-engine
• External issue URL: https://github.com/apache/arrow/issues/29843
• Assignee: Neal Richardson
• Reporter: Neal Richardson
• Created: 08/Oct/21 20:51
• Updated: 11/Jan/23 08:38
• Resolved: 20/Apr/22 00:16

Description

Following ARROW-13542. Any query that has a join or an aggregation currently has to first evaluate the query and hold it in memory before creating a Scanner to write it. We could improve that by using a WriteNode inside write_dataset() (and maybe that improves the other cases too, or at least allows us to delete some code).

Issue Links

• duplicates: ARROW-15517 [R] Use WriteNode in write_dataset()