Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13037

Refactor Azure Data Lake Store as an independent FileSystem

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: fs/adl
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hadoop now supports integration with Azure Data Lake as an alternative Hadoop-compatible file system. Please refer to the Hadoop site documentation of Azure Data Lake for details on usage and configuration.

      Description

      The jira proposes an improvement over HADOOP-12666 to remove webhdfs dependencies from the ADL file system client and build out a standalone client. At a high level, this approach would extend the Hadoop file system class to provide an implementation for accessing Azure Data Lake. The scheme used for accessing the file system will continue to be

      adl://<accountname>.azuredatalake.net/path/to/file.

      The Azure Data Lake Cloud Store will continue to provide a webHDFS rest interface. The client will access the ADLS store using WebHDFS Rest APIs provided by the ADLS store.

        Attachments

        1. HADOOP-13037.005.patch
          314 kB
          Chris Douglas
        2. HADOOP-13037.006.patch
          314 kB
          Vishwajeet Dusane
        3. HADOOP-13037-001.patch
          390 kB
          Vishwajeet Dusane
        4. HADOOP-13037-002.patch
          396 kB
          Vishwajeet Dusane
        5. HADOOP-13037-003.patch
          397 kB
          Vishwajeet Dusane
        6. HADOOP-13037-004.patch
          317 kB
          Vishwajeet Dusane
        7. HADOOP-13037 Proposal.pdf
          319 kB
          Vishwajeet Dusane

          Issue Links

            Activity

              People

              • Assignee:
                vishwajeet.dusane Vishwajeet Dusane
                Reporter:
                shrikant Shrikant Naidu
              • Votes:
                0 Vote for this issue
                Watchers:
                15 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: