Details

      Description

      This jira is for a ZK-based FailoverController daemon. The FailoverController is a separate daemon from the NN that does the following:

      • Initiates leader election (via ZK) when necessary
      • Performs health monitoring (aka failure detection)
      • Performs fail-over (standby to active and active to standby transitions)
      • Heartbeats to ensure the liveness

      It should have the same/similar interface as the Linux HA RM to aid pluggability.

      1. Failover_Controller.jpg
        138 kB
        Bikas Saha
      2. hdfs-2185.txt
        62 kB
        Todd Lipcon
      3. hdfs-2185.txt
        15 kB
        Todd Lipcon
      4. zkfc-design.pdf
        165 kB
        Todd Lipcon
      5. hdfs-2185.txt
        16 kB
        Todd Lipcon
      6. hdfs-2185.txt
        17 kB
        Todd Lipcon
      7. zkfc-design.tex
        15 kB
        Todd Lipcon
      8. zkfc-design.pdf
        232 kB
        Todd Lipcon
      9. zkfc-design.pdf
        234 kB
        Todd Lipcon
      10. hdfs-2185.txt
        18 kB
        Todd Lipcon
      11. zkfc-design.pdf
        243 kB
        Todd Lipcon

        Issue Links

          Activity

          Eli Collins created issue -
          Suresh Srinivas made changes -
          Field Original Value New Value
          Link This issue is related to HADOOP-7455 [ HADOOP-7455 ]
          Todd Lipcon made changes -
          Assignee Eli Collins [ eli ] Todd Lipcon [ tlipcon ]
          Todd Lipcon made changes -
          Link This issue incorporates HDFS-2681 [ HDFS-2681 ]
          Todd Lipcon made changes -
          Affects Version/s HA branch (HDFS-1623) [ 12317568 ]
          Target Version/s HA branch (HDFS-1623) [ 12317568 ]
          Component/s ha [ 12316609 ]
          Todd Lipcon made changes -
          Assignee Todd Lipcon [ tlipcon ] Bikas Saha [ bikassaha ]
          Todd Lipcon made changes -
          Assignee Bikas Saha [ bikassaha ] Todd Lipcon [ tlipcon ]
          Todd Lipcon made changes -
          Parent Issue HDFS-1623 [ HDFS-1623 ] HDFS-3042 [ HDFS-3042 ]
          Bikas Saha made changes -
          Attachment Failover_Controller.jpg [ 12516904 ]
          Todd Lipcon made changes -
          Attachment hdfs-2185.txt [ 12519761 ]
          Todd Lipcon made changes -
          Link This issue depends upon HADOOP-8206 [ HADOOP-8206 ]
          Todd Lipcon made changes -
          Summary HA: ZK-based FailoverController HA: HDFS portion of ZK-based FailoverController
          Todd Lipcon made changes -
          Attachment hdfs-2185.txt [ 12519906 ]
          Todd Lipcon made changes -
          Attachment zkfc-design.pdf [ 12519914 ]
          Todd Lipcon made changes -
          Attachment hdfs-2185.txt [ 12520029 ]
          Todd Lipcon made changes -
          Attachment hdfs-2185.txt [ 12520036 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Todd Lipcon made changes -
          Affects Version/s 0.24.0 [ 12317653 ]
          Affects Version/s 0.23.3 [ 12320052 ]
          Affects Version/s HA branch (HDFS-1623) [ 12317568 ]
          Target Version/s HA branch (HDFS-1623) [ 12317568 ] 0.24.0, 0.23.3 [ 12317653, 12320052 ]
          Todd Lipcon made changes -
          Attachment zkfc-design.tex [ 12520046 ]
          Todd Lipcon made changes -
          Attachment zkfc-design.pdf [ 12520047 ]
          Todd Lipcon made changes -
          Comment [ -1 overall. Here are the results of testing the latest attachment
            http://issues.apache.org/jira/secure/attachment/12520047/zkfc-design.pdf
            against trunk revision .

              +1 @author. The patch does not contain any @author tags.

              -1 tests included. The patch doesn't appear to include any new or modified tests.
                                  Please justify why no new tests are needed for this patch.
                                  Also please list what manual steps were performed to verify this patch.

              -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2104//console

          This message is automatically generated. ]
          Todd Lipcon made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 0.23.3, 0.24.0 [ 12320052, 12317653 ] 0.24.0, 0.23.3 [ 12317653, 12320052 ]
          Todd Lipcon made changes -
          Target Version/s 0.23.3, 0.24.0 [ 12320052, 12317653 ] Auto failover (HDFS-3042) [ 12320351 ]
          Component/s auto-failover [ 12317907 ]
          Todd Lipcon made changes -
          Attachment zkfc-design.pdf [ 12520870 ]
          Todd Lipcon made changes -
          Attachment hdfs-2185.txt [ 12521087 ]
          Todd Lipcon made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s Auto failover (HDFS-3042) [ 12320351 ]
          Resolution Fixed [ 1 ]
          Todd Lipcon made changes -
          Attachment zkfc-design.pdf [ 12521279 ]
          Allen Wittenauer made changes -
          Affects Version/s 2.0.0-alpha [ 12320353 ]
          Affects Version/s 0.24.0 [ 12317653 ]

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Eli Collins
            • Votes:
              0 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development