Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-1213

[Python] Enable s3fs to be used with ParquetDataset and reader/writer functions

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: Python

      Description

      Pyarrow dataset function can't read from s3 using s3fs as the filesystem. Is there a way we can add the support for read from s3 based on partitioned files ?

      I am trying to address the problem mentioned in the stackoverflow link :
      https://stackoverflow.com/questions/45082832/how-to-read-partitioned-parquet-files-from-s3-using-pyarrow-in-python

        Attachments

          Activity

            People

            • Assignee:
              wesmckinn Wes McKinney
              Reporter:
              yackoa Yacko
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: