[ZOOKEEPER-1080] Provide a Leader Election framework based on Zookeeper recipe

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.3.2
• Fix Version/s: 3.5.0
• Component/s: contrib
• Assignee: Hari A V
• Reporter: Hari A V
• Created: 30/May/11 13:24
• Updated: 17/May/14 12:03
• Resolved: 17/May/14 12:03

Description

Currently Hadoop components such as NameNode and JobTracker are single point of failure.
If Namenode or JobTracker goes down, there service will not be available until they are up and running again. If there was a Standby Namenode or JobTracker available and ready to serve when Active nodes go down, we could have reduced the service down time. Hadoop already provides a Standby Namenode implementation which is not fully a "hot" Standby.
The common problem to be addressed in any such Active-Standby cluster is Leader Election and Failure detection. This can be done using Zookeeper as mentioned in the Zookeeper recipes.
http://zookeeper.apache.org/doc/r3.3.3/recipes.html

Leader Election Service (LES)

Any Node who wants to participate in Leader Election can use this service. They should start the service with required configurations. The service will notify the nodes whether they should be started as Active or Standby mode. Also they intimate any changes in the mode at runtime. All other complexities can be handled internally by the LES.

Issue Links

• blocks: HIVE-2254 Provide an automatic recovery feature for Hive Server in case of failure
• blocks: HDFS-2124 Namenode HA using Backup Namenode as Hot Standby
• blocks: MAPREDUCE-2648 High Availability for JobTracker
• duplicates: ZOOKEEPER-1095 Simple leader election recipe
• relates to: HDFS-1973 HA: HDFS clients must handle namenode failover and switch over to the new active namenode.