Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2046

[Python] Add support for PEP519 - pathlib and similar objects

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Python

      Description

      Currently `pyarrow` doesn't seem to support reading from `pathlib.Path` or similar objects. PEP519 introduced `_fspath_` which could be used to transform any `Path` like object to a string.

      Pandas has a sample implementation, though I think a simpler implementation of it could be used.

       

      import pathlib
      import pandas as pd
      import pyarrow as pa
      import pyarrow.parquet as pq
      
      df = pd.DataFrame({
       'Foo': ['A', 'A', 'B', 'B', 'C'],
       'Bar': ['A1', 'A2', 'B2', 'D3', ''],
      })
      
      test_dir = pathlib.Path(__file__).parent / 'test'
      test_dir.mkdir(parents=True, exist_ok=True)
      table = pa.Table.from_pandas(df)
      path = test_dir / 'file1.parquet'
      
      # Doesn't work
      pq.write_table(table, path)
      
      # Works
      pq.write_table(table, str(path))
      

       

      https://github.com/apache/arrow/issues/1522

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                pitrou Antoine Pitrou
                Reporter:
                vmuriart Victor Uriarte
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: