[ARROW-1213] [Python] Enable s3fs to be used with ParquetDataset and reader/writer functions

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/17002
• Assignee: Wes McKinney
• Reporter: Yacko
• Created: 13/Jul/17 14:00
• Updated: 11/Jan/23 07:13
• Resolved: 31/Jul/17 22:47

Description

Pyarrow dataset function can't read from s3 using s3fs as the filesystem. Is there a way we can add the support for read from s3 based on partitioned files ?

I am trying to address the problem mentioned in the stackoverflow link :
https://stackoverflow.com/questions/45082832/how-to-read-partitioned-parquet-files-from-s3-using-pyarrow-in-python

Issue Links

• links to: GitHub Pull Request #916
• links to: GitHub Pull Request #916