Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10467

Router-based HDFS federation

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.1
    • 2.9.0, 3.0.0
    • fs
    • Reviewed
    • Hide
      HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces.
      This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client.
      This simplifies access to a federated cluster for existing HDFS clients.

      See HDFS-10467 and the HDFS Router-based Federation documentation for more details.
      Show
      HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces. This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client. This simplifies access to a federated cluster for existing HDFS clients. See HDFS-10467 and the HDFS Router-based Federation documentation for more details.

    Description

      Add a Router to provide a federated view of multiple HDFS clusters.

      Attachments

        1. HDFS Router Federation.pdf
          944 kB
          Íñigo Goiri
        2. HDFS-Router-Federation-Prototype.patch
          616 kB
          Íñigo Goiri
        3. HDFS-10467.PoC.patch
          664 kB
          Íñigo Goiri
        4. HDFS-10467.PoC.001.patch
          725 kB
          Íñigo Goiri
        5. HDFS-10467.002.patch
          897 kB
          Íñigo Goiri

        Issue Links

          depends upon

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13144 Enhancing IPC client throughput via multiple connections per user

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          incorporates

          Sub-task - The sub-task of the issue HDFS-10629 Federation Router

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HDFS-12615 Router-based HDFS federation phase 2

          • Major - Major loss of function.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HDFS-6648 Order of namenodes in ConfiguredFailoverProxyProvider is undefined

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-12077 Provide a multi-URI replication Inode for ViewFs

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-2915 Enable YARN RM scale out via federation using multiple RM's

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-7257 A client side mount table to give per-application/per-job file system view

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue HDFS-12284 RBF: Support for Kerberos authentication

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-12312 Rebasing HDFS-10467 (2)

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue HDFS-12620 Backporting HDFS-10467 to branch-2

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-13378 Common features between YARN and HDFS Router-based federation

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14939 Update project release notes with HDFS-10467 for 3.0.0

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              elgoiri Íñigo Goiri
              elgoiri Íñigo Goiri
              Votes:
              0 Vote for this issue
              Watchers:
              79 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: