Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6440

Support more than 2 NameNodes

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.4.0
    • 3.0.0-alpha1
    • auto-failover, ha, namenode
    • None
    • Incompatible change, Reviewed
    • This feature adds support for running additional standby NameNodes, which provides additional fault-tolerance. It is designed for a total of 3-5 NameNodes.

    Description

      Most of the work is already done to support more than 2 NameNodes (one active, one standby). This would be the last bit to support running multiple standby NameNodes; one of the standbys should be available for fail-over.

      Mostly, this is a matter of updating how we parse configurations, some complexity around managing the checkpointing, and updating a whole lot of tests.

      Attachments

        1. Multiple-Standby-NameNodes_V1.pdf
          217 kB
          Jesse Yates
        2. hdfs-multiple-snn-trunk-v0.patch
          171 kB
          Jesse Yates
        3. hdfs-6440-trunk-v8.patch
          529 kB
          Jesse Yates
        4. hdfs-6440-trunk-v7.patch
          529 kB
          Jesse Yates
        5. hdfs-6440-trunk-v6.patch
          527 kB
          Jesse Yates
        6. hdfs-6440-trunk-v5.patch
          526 kB
          Jesse Yates
        7. hdfs-6440-trunk-v4.patch
          188 kB
          Jesse Yates
        8. hdfs-6440-trunk-v3.patch
          178 kB
          Jesse Yates
        9. hdfs-6440-trunk-v1.patch
          171 kB
          Jesse Yates
        10. hdfs-6440-trunk-v1.patch
          171 kB
          Jesse Yates
        11. hdfs-6440-cdh-4.5-full.patch
          138 kB
          Jesse Yates

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. HDFS-9659 EditLogTailerThread to Active Namenode RPC should timeout

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-9787 SNNs stop uploading FSImage to ANN once isPrimaryCheckPointer changed to false.

        • Major - Major loss of function.
        • Resolved
        causes

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-15684 triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-14305 Serial number in BlockTokenSecretManager could overlap between different namenodes

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-14883 NPE when the second SNN is starting

        • Major - Major loss of function.
        • Resolved
        is related to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-14646 Standby NameNode should not upload fsimage to an inappropriate NameNode.

        • Major - Major loss of function.
        • Patch Available

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-13933 Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-14201 Ability to disallow safemode NN to become active

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-14378 Simplify the design of multiple NN and both logic of edit log roll and checkpoint

        • Major - Major loss of function.
        • Patch Available
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HDFS-10536 Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll method.

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-13900 NameNode: Unable to trigger a roll of the active NN

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-6648 Order of namenodes in ConfiguredFailoverProxyProvider is undefined

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-14793 BlockTokenSecretManager should LOG block token range it operates on.

        • Major - Major loss of function.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HDFS-9423 Fix intermittent failure of TestEditLogTailer

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. HDFS-12943 Consistent Reads from Standby Node

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-14205 Backport HDFS-6440 to branch-2

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HDFS-14969 Fix HDFS client unnecessary failover log printing

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jesse_yates Jesse Yates
            jesse_yates Jesse Yates
            Votes:
            0 Vote for this issue
            Watchers:
            65 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment