Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2013

[Python] Add AzureDataLakeFilesystem to be used with ParquetDataset and reader/writer functions

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Minor
    • Resolution: Later
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Python
    • Labels:

      Description

      Similar to https://issues.apache.org/jira/browse/ARROW-1213, it would be great to add AzureDLFileSystem as a supported filesystem in ParquetDataset.

      Example:

      from azure.datalake.store import AzureDLFileSystem
      fs = AzureDLFileSystem(token=token, store_name=store_name)
      dataset = pq.ParquetDataset(file_list, filesystem=fs)

      Throws:

      IOError: Unrecognized filesystem: <class 'azure.datalake.store.core.AzureDLFileSystem'>

      Azures github:
      https://github.com/Azure/azure-data-lake-store-python

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              npezolano Nicholas Pezolano
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: