[ARROW-16612] [R] Fix compression inference from filename

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 8.0.0
• Fix Version/s: 9.0.0
• Component/s: R
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/31966
• Assignee: Neal Richardson
• Reporter: Sam Albers
• Created: 18/May/22 22:12
• Updated: 11/Jan/23 11:45
• Resolved: 27/Jul/22 11:02

Description

Right now arrow will silently write a file with a .gz extension to CompressedOutputStream rather than passing the compression option to the parquet writer itself. The internal detect_compression() function detects the extension and that is what passes it off incorrectly. However it only fails at the read_parquet stage which could lead to confusion.

library(arrow, warn.conflicts = FALSE) 
tf <- tempfile(fileext = ".parquet.gz") 
write_parquet(data.frame(x = 1:5), tf, compression = "gzip", compression_level = 5) read_parquet(tf) 
#> Error: file must be a "RandomAccessFile"

Issue Links

• fixes: ARROW-16619 [R] Support compression + R connection (URL with .gz file)
• links to: GitHub Pull Request #13625