Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10467

Router-based HDFS federation

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.1
    • 2.9.0, 3.0.0
    • fs
    • Reviewed
    • Hide
      HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces.
      This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client.
      This simplifies access to a federated cluster for existing HDFS clients.

      See HDFS-10467 and the HDFS Router-based Federation documentation for more details.
      Show
      HDFS Router-based Federation adds a RPC routing layer that provides a federated view of multiple HDFS namespaces. This is similar to the existing ViewFS and HDFS federation functionality, except the mount table is managed on the server-side by the routing layer rather than on the client. This simplifies access to a federated cluster for existing HDFS clients. See HDFS-10467 and the HDFS Router-based Federation documentation for more details.

    Description

      Add a Router to provide a federated view of multiple HDFS clusters.

      Attachments

        1. HDFS-10467.002.patch
          897 kB
          Íñigo Goiri
        2. HDFS-10467.PoC.001.patch
          725 kB
          Íñigo Goiri
        3. HDFS-10467.PoC.patch
          664 kB
          Íñigo Goiri
        4. HDFS Router Federation.pdf
          944 kB
          Íñigo Goiri
        5. HDFS-Router-Federation-Prototype.patch
          616 kB
          Íñigo Goiri

        Issue Links

        1.
        Federation Router Sub-task Resolved Jason Kace Actions
        2.
        Federation Router RPC server Sub-task Resolved Íñigo Goiri Actions
        3.
        Federation State Store Driver API Sub-task Resolved Jason Kace Actions
        4.
        Federation State Store Interface API Sub-task Resolved Jason Kace Actions
        5.
        Federation State Store FS Implementation Sub-task Resolved Jason Kace Actions
        6.
        Federation State Store ZooKeeper implementation Sub-task Resolved Jason Kace Actions
        7.
        Federation Membership State Store internal API Sub-task Resolved Jason Kace Actions
        8.
        Federation Namenode Heartbeat Sub-task Resolved Íñigo Goiri Actions
        9.
        Federation Mount Table State Store internal API Sub-task Resolved Íñigo Goiri Actions
        10.
        Federation admin tool Sub-task Resolved Íñigo Goiri Actions
        11.
        [Documentation] Router-based federation documentation Sub-task Resolved Íñigo Goiri Actions
        12.
        Rebasing HDFS-10467 Sub-task Resolved Íñigo Goiri Actions
        13.
        Rebasing HDFS-10467 (2) Sub-task Resolved Íñigo Goiri Actions
        14.
        Fixing compilation issue with BanDuplicateClasses Sub-task Resolved Íñigo Goiri Actions
        15.
        Federation Metrics Sub-task Resolved Íñigo Goiri Actions
        16.
        Rebasing HDFS-10467 After HDFS-12269 and HDFS-12218 Sub-task Resolved Íñigo Goiri Actions
        17.
        Fixing TestNamenodeHeartbeat and support non-HA Sub-task Resolved Íñigo Goiri Actions
        18.
        [Documentation] Adding configuration keys for the Router Sub-task Resolved Íñigo Goiri Actions
        19.
        Rebasing HDFS-10467 after HDFS-12447 Sub-task Resolved Íñigo Goiri Actions
        20.
        Federation UI Sub-task Resolved Íñigo Goiri Actions
        21.
        Rename Router tooling Sub-task Resolved Íñigo Goiri Actions
        22.
        Backporting HDFS-10467 to branch-2 Sub-task Resolved Íñigo Goiri Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            elgoiri Íñigo Goiri
            elgoiri Íñigo Goiri
            Votes:
            0 Vote for this issue
            Watchers:
            84 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment