[ARROW-5131] [Python] Add Azure Datalake Filesystem Gen1 Wrapper for pyarrow

• Type: Wish
• Status: Closed
• Priority: Minor
• Resolution: Later
• Affects Version/s: 0.12.1
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/21615
• Assignee: Unassigned
• Reporter: Gregory Hayes
• Created: 06/Apr/19 15:08
• Updated: 11/Jan/23 07:38
• Resolved: 18/Sep/19 15:23

Description

The current pyarrow package can only read parquet files that have been written to Gen1 Azure Datalake using the fastparquet engine. This only works if the dask-adlfs package is explicitly installed and imported. I've added a method to the dask-adlfs package, found here, and issued a PR for that change. To support this capability, added an ADLFSWrapper to filesystem.py file.

Issue Links

• links to: GitHub Pull Request #4121