Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-8942

[R] Detect compression in reading CSV/JSON

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.0.0
    • R

    Description

      Hi all,

      Apologises if this has already been covered by another ticket. Is it possible for arrow to read in compress delimited files (for example gzip)?

      Currently I get an error when trying to read in a compressed delimited file:

       

      vroom::vroom_write(iris, "iris.csv.gz", delim = ",")
      arrow::read_csv_arrow("iris.csv.gz")
      
      # Error in csv__TableReader_Read(self) :
      # Invalid: CSV parse error: Expected 1 columns, got 4

      however it can be read in by vroom and readr:

      vroom::vroom("iris.csv.gz")
      readr::read_csv("iris.csv.gz")
      

       

       

       

       

      Attachments

        Issue Links

          Activity

            People

              npr Neal Richardson
              larefly Dyfan Jones
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m