Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14805

Mounting external stores in HDFS on-the-fly

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Provided storage (HDFS-9806) allows HDFS to address data in external storage systems, including cloud stores. Data mounted in this manner, seamlessly, appears to be part of HDFS for applications/clients. The external data can also be cached by HDFS on local disks and SSDs, accelerating remote data reads (HDFS-13069).

      However, Provided storage was originally targeted at ephemeral HDFS deployments in the cloud (e.g., Azure HDInsight). Long running HDFS clusters are common in many other scenarios which can benefit from accessing data in remote stores. This JIRA targets such scenarios and aims to provide the ability to:
      (a) Dynamically mount external stores in a HDFS cluster while supporting high availability.
      (b) Mount multiple remote stores simultaneously.
      (c) Reduce deployment overheads and simplify usability of Provided storage.

        Attachments

        1. dynamic-mounts-in-hdfs.pdf
          949 kB
          Virajith Jalaparti

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                virajith Virajith Jalaparti
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: