[AMBARI-5840] Ambari recomission of datanodes will not work if the active namenode is "nn2"

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.5.1
• Fix Version/s: 1.6.1
• Component/s: ambari-agent

Environment

java 1.6 update 31
Centos 6.4
Not specific to Ambari, but ambari 1.5.1 this was reproduced.

• Assignee: Jonathan Hurley
• Reporter: Jonathan Hurley
• Created: 21/May/14 15:53
• Updated: 22/May/14 17:58
• Resolved: 22/May/14 17:58

Description

PROBLEM: Ambari recomission of datanodes will not work if the active namenode is "nn2"
BUSINESS IMPACT: This will cause a node to not ever be recomissioned if the nn2 machine is the active namenode. Basically, any change that requires a refreshnodes command, will not be applied if nn2 is the active namenode.
STEPS TO REPRODUCE:
1. Create a new cluster with HA for namenode
2. Decomission a datanode
3. Failover the namenode, so the node that corresponds to nn2 is the active namenode.
for example in hdfs-site.xml:
<property>
<name>dfs.namenode.rpc-address.cluster01.nn2</name>
<value>master02.server.com:8020</value>
</property>
the active namenode host would be master02.server.com.
3. Now try to recomission the node.
4. when the process is done as per ambari, execute a hdfs dfsadmin -report and grep for "Decommissioned". You should see that machine you just try to recommission as decomissioned.
ACTUAL BEHAVIOR: The recomission process does not work in ambari if nn2 is the active namenode.
EXPECTED BEHAVIOR: The recomission process should work.
WORKAROUND: Use the command:
hdfs dfsadmin -fs hdfs://<namenodehostactive>:8020 -refreshNodes
This will refreshnodes properly for the active namenode.

Issue Links

• links to: ReviewBoard