[ARROW-2046] [Python] Add support for PEP519 - pathlib and similar objects

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/18025
• Assignee: Antoine Pitrou
• Reporter: Victor Uriarte
• Created: 27/Jan/18 22:31
• Updated: 11/Jan/23 07:18
• Resolved: 28/Feb/18 19:49

Description

Currently `pyarrow` doesn't seem to support reading from `pathlib.Path` or similar objects. PEP519 introduced `_fspath_` which could be used to transform any `Path` like object to a string.

Pandas has a sample implementation, though I think a simpler implementation of it could be used.

 

import pathlib
import pandas as pd
import pyarrow as pa
import pyarrow.parquet as pq

df = pd.DataFrame({
 'Foo': ['A', 'A', 'B', 'B', 'C'],
 'Bar': ['A1', 'A2', 'B2', 'D3', ''],
})

test_dir = pathlib.Path(__file__).parent / 'test'
test_dir.mkdir(parents=True, exist_ok=True)
table = pa.Table.from_pandas(df)
path = test_dir / 'file1.parquet'

# Doesn't work
pq.write_table(table, path)

# Works
pq.write_table(table, str(path))

 

https://github.com/apache/arrow/issues/1522

 

 

Issue Links

• links to: GitHub Pull Request #1675