[HDFS-13433] webhdfs requests can be routed incorrectly in federated cluster

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.1.1, 3.0.3
• Component/s: webhdfs
• Hadoop Flags: Reviewed
• Assignee: Arpit Agarwal
• Reporter: Arpit Agarwal
• Created: 12/Apr/18 00:40
• Updated: 31/Jan/19 04:30
• Resolved: 23/Apr/18 17:16

Description

In the following HA+Federated setup with two nameservices ns1 and ns2:

1. ns1 -> namenodes nn1, nn2
2. ns2 -> namenodes nn3, nn4
3. fs.defaultFS is hdfs://ns1.

A webhdfs request issued to nn3/nn4 will be routed to ns1. This is because setClientNamenodeAddress initializes NameNode#clientNamenodeAddress using fs.defaultFS before the config is overriden.

Issue Links

• duplicates: HDFS-13138 webhdfs of federated namenode does not work properly
• is related to: HDFS-6329 WebHdfs does not work if HA is enabled on NN but logical URI is not configured.