[ARROW-1555] [Python] write_to_dataset on s3

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 0.7.0
• Fix Version/s: 0.8.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/17570
• Assignee: Benjamin Goldberg
• Reporter: Young-Jun Ko
• Created: 19/Sep/17 13:12
• Updated: 11/Jan/23 07:15
• Resolved: 27/Oct/17 02:59

Description

When writing a arrow table to s3, I get an NotImplemented Exception.
The root cause is in _ensure_filesystem and can be reproduced as follows:

import pyarrow
import pyarrow.parquet as pqa
import s3fs

s3 = s3fs.S3FileSystem()

pqa._ensure_filesystem(s3).exists("anything")

It appears that the S3FSWrapper that is instantiated in _ensure_filesystem does not expose the exist method of s3.

Issue Links

• links to: GitHub Pull Request #1240