Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10467

Router-based HDFS federation

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.1
    • Fix Version/s: 2.9.0, 3.0.0
    • Component/s: fs
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces.
      This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client.
      This simplifies access to a federated cluster for existing HDFS clients.

      See HDFS-10467 and the HDFS Router-based Federation documentation for more details.
      Show
      HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces. This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client. This simplifies access to a federated cluster for existing HDFS clients. See HDFS-10467 and the HDFS Router-based Federation documentation for more details.

      Description

      Add a Router to provide a federated view of multiple HDFS clusters.

        Attachments

        1. HDFS Router Federation.pdf
          944 kB
          Íñigo Goiri
        2. HDFS-Router-Federation-Prototype.patch
          616 kB
          Íñigo Goiri
        3. HDFS-10467.PoC.patch
          664 kB
          Íñigo Goiri
        4. HDFS-10467.PoC.001.patch
          725 kB
          Íñigo Goiri
        5. HDFS-10467.002.patch
          897 kB
          Íñigo Goiri

        Issue Links

        1.
        Federation Router Sub-task Resolved Jason Kace Actions
        2.
        Federation Router RPC server Sub-task Resolved Íñigo Goiri Actions
        3.
        Federation State Store Driver API Sub-task Resolved Jason Kace Actions
        4.
        Federation State Store Interface API Sub-task Resolved Jason Kace Actions
        5.
        Federation State Store FS Implementation Sub-task Resolved Jason Kace Actions
        6.
        Federation State Store ZooKeeper implementation Sub-task Resolved Jason Kace Actions
        7.
        Federation Membership State Store internal API Sub-task Resolved Jason Kace Actions
        8.
        Federation Namenode Heartbeat Sub-task Resolved Íñigo Goiri Actions
        9.
        Federation Mount Table State Store internal API Sub-task Resolved Íñigo Goiri Actions
        10.
        Federation admin tool Sub-task Resolved Íñigo Goiri Actions
        11.
        [Documentation] Router-based federation documentation Sub-task Resolved Íñigo Goiri Actions
        12.
        Rebasing HDFS-10467 Sub-task Resolved Íñigo Goiri Actions
        13.
        Rebasing HDFS-10467 (2) Sub-task Resolved Íñigo Goiri Actions
        14.
        Fixing compilation issue with BanDuplicateClasses Sub-task Resolved Íñigo Goiri Actions
        15.
        Federation Metrics Sub-task Resolved Íñigo Goiri Actions
        16.
        Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218 Sub-task Resolved Íñigo Goiri Actions
        17.
        Fixing TestNamenodeHeartbeat and support non-HA Sub-task Resolved Íñigo Goiri Actions
        18.
        [Documentation] Adding configuration keys for the Router Sub-task Resolved Íñigo Goiri Actions
        19.
        Rebasing HDFS-10467 after HDFS-12447 Sub-task Resolved Íñigo Goiri Actions
        20.
        Federation UI Sub-task Resolved Íñigo Goiri Actions
        21.
        Rename Router tooling Sub-task Resolved Íñigo Goiri Actions
        22.
        Backporting HDFS-10467 to branch-2 Sub-task Resolved Íñigo Goiri Actions

          Activity

            People

            • Assignee:
              elgoiri Íñigo Goiri
              Reporter:
              elgoiri Íñigo Goiri

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment