[ARROW-16144] [R] Write compressed data streams (particularly over S3)

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 7.0.0
• Fix Version/s: 9.0.0
• Component/s: R
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31551
• Assignee: Sam Albers
• Reporter: Carl Boettiger
• Created: 07/Apr/22 17:03
• Updated: 11/Jan/23 11:42
• Resolved: 18/May/22 21:25

Description

The python bindings have `CompressedOutputStream`, but  I don't see how we can do this on the R side (e.g. with `write_csv_arrow()`).  It would be wonderful if we could both read and write compressed streams, particularly for CSV and particularly for remote filesystems, where this can provide considerable performance improvements. 

(For comparison, readr will write a compressed stream automatically based on the extension for the given filename, e.g. `readr::write_csv(data, "file.csv.gz")` or `write_csv("data.file.xz")`  )

Issue Links

• links to: GitHub Pull Request #13183