[HDFS-6100] DataNodeWebHdfsMethods does not failover in HA mode

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.4.0
• Component/s: ha
• Target Version/s: 2.4.0
• Hadoop Flags: Reviewed
• Assignee: Haohui Mai
• Reporter: Arpit Gupta
• Created: 13/Mar/14 00:13
• Updated: 10/Apr/14 13:11
• Resolved: 19/Mar/14 17:36

Description

In DataNodeWebHdfsMethods, the code creates a DFSClient to connect to the NN, so that it can access the files in the cluster. DataNodeWebHdfsMethods relies on the address passed in the URL to locate the NN. This implementation has two problems:

1. The DFSClient only knows about the current active NN, thus it does not support failover.
2. The delegation token is based on the active NN, therefore the DFSClient will fail to authenticate of the standby NN in secure HA setup.

Currently the parameter namenoderpcaddress in the URL stores the host-ip pair that corresponds to the active NN. To fix this bug, this jira proposes to store the name service id in the parameter in HA setup (yet the parameter stays the same in non-HA setup).

Issue Links

• breaks: HDFS-6140 WebHDFS cannot create a file with spaces in the name after HA failover changes.