[ARROW-15104] write_parquet() / write_csv_arrow() cannot stream a dataset object back to S3

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 6.0.1
• Component/s: R
• External issue URL: https://github.com/apache/arrow/issues/30615
• Assignee: Unassigned
• Reporter: Carl Boettiger
• Created: 14/Dec/21 16:53
• Updated: 11/Jan/23 08:44
• Resolved: 14/Dec/21 16:58

Description

It seems we cannot send a dataset (table) object opened with `open_dataset()` into `write_parquet()` or `write_csv_arrow()`.  I realize that the former is a 'multi-file' representation and the latter two are not, but as both represent a single table (single schema) it seems like this should still work.  (It also looks like write_csv_arrow() gives us more control over formatting details than write_dataset() )

Issue Links

• duplicates: ARROW-14428 [R] [C++] Allow me to write_parquet() from an arrow_dplyr_query
• duplicates: ARROW-15040 [R] Enable write_csv_arrow to take Dataset or arrow_dplyr_query as input