[ARROW-10998] [C++] Filesystems: detect if URI is passed where a file path is required and raise informative error

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.0.0
• Component/s: C++, Python
• Labels: filesystem pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18430
• Assignee: Antoine Pitrou
• Reporter: Joris Van den Bossche
• Created: 21/Dec/20 14:11
• Updated: 11/Jan/23 08:16
• Resolved: 20/Dec/21 19:55

Description

Currently, when passing a URI to a filesystem method (except for from_uri) or other functions that accept a filesystem object, you can get a rather cryptic error message (eg in this case about "No response body" for S3, in the example below).

Ideally, the filesystem object knows its own prefix "scheme", and so can detect if a user is passing a URI instead of file path, and we can provide a nicer error message.

Example with S3:

>>> from pyarrow.fs import S3FileSystem
>>> fs = S3FileSystem(region="us-east-2")
>>> fs.get_file_info('s3://ursa-labs-taxi-data/2016/01/')
...
OSError: When getting information for key '/ursa-labs-taxi-data/2016/01' in bucket 's3:': AWS Error [code 100]: No response body.

>>> import pyarrow.parquet as pq
>>> table = pq.read_table('s3://ursa-labs-taxi-data/2016/01/data.parquet', filesystem=fs)
...
OSError: When getting information for key '/ursa-labs-taxi-data/2016/01/data.parquet' in bucket 's3:': AWS Error [code 100]: No response body.

With a local filesystem, you actually get a not found file:

Unable to find source-code formatter for language: python. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml

>>> fs = LocalFileSystem()
>>> fs.get_file_info("file:///home")
<FileInfo for 'file:///home': type=FileType.NotFound>

cc apitrou

Issue Links

• is related to: ARROW-10937 [Python] ArrowInvalid error on reading partitioned parquet files from S3 (arrow-2.0.0)
• links to: GitHub Pull Request #11997