[HDFS-6648] Order of namenodes in ConfiguredFailoverProxyProvider is undefined

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Component/s: ha, hdfs-client
• Hadoop Flags: Reviewed
• Assignee: Íñigo Goiri
• Reporter: Rafal Wojdyla
• Created: 09/Jul/14 12:48
• Updated: 12/May/20 15:11
• Resolved: 21/Mar/17 00:21

Description

In org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider, in the constructor, there's a map <nameservice : < service-id : service-rpc-address > > (DFSUtil.getHaNnRpcAddresses). It's a LinkedHashMap of HashMaps. The order is kept for nameservices. Then to find active namenode, for nameservice, we get HashMap of <service-id : service-rpc-address > for requested nameservice (taken from URI request), And for this HashMap we get values - order of this collection is not strictly defined! In the code:

Collection<InetSocketAddress> addressesOfNns = addressesInNN.values(); 

And then we put these values (in not defined order) into ArrayList of proxies, and then in getProxy we start from first proxy in the list and failover to next if needed.

It would make sense for ConfiguredFailoverProxyProvider to keep order of proxies/namenodes defined in hdfs-site.xml.

Issue Links

• is related to: HDFS-6440 Support more than 2 NameNodes
• relates to: HDFS-12976 Introduce ObserverReadProxyProvider
• relates to: HDFS-10467 Router-based HDFS federation
• relates to: HDFS-13653 Make dfs.client.failover.random.order a per nameservice configuration
• relates to: HDFS-15350 Set dfs.client.failover.random.order to true as default