[ARROW-8942] [R] Detect compression in reading CSV/JSON

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: R
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/25070
• Assignee: Neal Richardson
• Reporter: Dyfan Jones
• Created: 26/May/20 11:37
• Updated: 11/Jan/23 08:03
• Resolved: 16/Jun/20 16:05

Description

Hi all,

Apologises if this has already been covered by another ticket. Is it possible for arrow to read in compress delimited files (for example gzip)?

Currently I get an error when trying to read in a compressed delimited file:

 

vroom::vroom_write(iris, "iris.csv.gz", delim = ",")
arrow::read_csv_arrow("iris.csv.gz")

# Error in csv__TableReader_Read(self) :
# Invalid: CSV parse error: Expected 1 columns, got 4

however it can be read in by vroom and readr:

vroom::vroom("iris.csv.gz")
readr::read_csv("iris.csv.gz")

 

 

 

 

Issue Links

• links to: GitHub Pull Request #7423