[HDFS-6440] Support more than 2 NameNodes

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0-alpha1
• Component/s: auto-failover, ha, namenode
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: This feature adds support for running additional standby NameNodes, which provides additional fault-tolerance. It is designed for a total of 3-5 NameNodes.
• Assignee: Jesse Yates
• Reporter: Jesse Yates
• Created: 20/May/14 22:51
• Updated: 08/Nov/19 16:25
• Resolved: 24/Jun/15 00:29

Description

Most of the work is already done to support more than 2 NameNodes (one active, one standby). This would be the last bit to support running multiple standby NameNodes; one of the standbys should be available for fail-over.

Mostly, this is a matter of updating how we parse configurations, some complexity around managing the checkpointing, and updating a whole lot of tests.

Issue Links

• breaks: HDFS-9659 EditLogTailerThread to Active Namenode RPC should timeout
• breaks: HDFS-9787 SNNs stop uploading FSImage to ANN once isPrimaryCheckPointer changed to false.
• causes: HADOOP-15684 triggerActiveLogRoll stuck on dead name node, when ConnectTimeoutException happens.
• causes: HDFS-14305 Serial number in BlockTokenSecretManager could overlap between different namenodes
• causes: HDFS-14883 NPE when the second SNN is starting
• is related to: HDFS-14646 Standby NameNode should not upload fsimage to an inappropriate NameNode.
• is related to: HADOOP-13933 Add haadmin -getAllServiceState option to get the HA state of all the NameNodes/ResourceManagers
• is related to: HDFS-14201 Ability to disallow safemode NN to become active
• is related to: HDFS-14378 Simplify the design of multiple NN and both logic of edit log roll and checkpoint
• relates to: HDFS-10536 Standby NN can not trigger log roll after EditLogTailer thread failed 3 times in EditLogTailer.triggerActiveLogRoll method.
• relates to: HDFS-13900 NameNode: Unable to trigger a roll of the active NN
• relates to: HDFS-6648 Order of namenodes in ConfiguredFailoverProxyProvider is undefined
• relates to: HDFS-14793 BlockTokenSecretManager should LOG block token range it operates on.
• relates to: HDFS-9423 Fix intermittent failure of TestEditLogTailer
• relates to: HDFS-12943 Consistent Reads from Standby Node
• relates to: HDFS-14205 Backport HDFS-6440 to branch-2
• relates to: HDFS-14969 Fix HDFS client unnecessary failover log printing