[ARROW-4559] [Python] pyarrow can't read/write filenames with special characters

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: Python
• Labels: pull-request-available
• Environment: $ python3 --version ; Python 3.6.6 ; $ pip3 freeze | grep -Ei 'pyarrow|pandas' ; pandas==0.24.1 ; pyarrow==0.12.0
• External issue URL: https://github.com/apache/arrow/issues/21106
• Assignee: Antoine Pitrou
• Reporter: Jean-Christophe Petkovich
• Created: 13/Feb/19 05:34
• Updated: 11/Jan/23 07:34
• Resolved: 21/Feb/19 10:07

Description

When writing or reading files to or from paths that have special characters in them, (e.g., "#"), pyarrow returns an error: 

OSError: Passed non-file path...

This is a consequence of the following line:
https://github.com/apache/arrow/blob/master/python/pyarrow/filesystem.py#L416

File-paths will be parsed as URIs, which will give strange results for filepaths like: "bad # actor.parquet":

ParseResult(scheme='', netloc='', path='/tmp/bad ', params='', query='', fragment='actor.parquet')

This is trivial to reproduce with the following code which uses the `pd.to_parquet` and `pd.read_parquet` interfaces:

import pandas as pd
x = pd.DataFrame({"a": [1,2,3]})
x.to_parquet("bad # actor.parquet")
x.read_parquet("bad # actor.parquet")

Issue Links

• is duplicated by: ARROW-4311 [Python] Regression on pq.ParquetWriter incorrectly handling source string
• supercedes: ARROW-4480 [Python] Drive letter removed when writing parquet file
• links to: GitHub Pull Request #3718