[HDFS-13388] RequestHedgingProxyProvider calls multiple configured NNs all the time

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1, 3.0.3
• Component/s: hdfs-client
• Hadoop Flags: Reviewed
• Assignee: Jinglun
• Reporter: Jinglun
• Created: 03/Apr/18 11:11
• Updated: 21/May/18 20:21
• Resolved: 23/Apr/18 02:50

Description

In HDFS-7858 RequestHedgingProxyProvider was designed to "first simultaneously call multiple configured NNs to decide which is the active Namenode and then for subsequent calls it will invoke the previously successful NN ." But the current code call multiple configured NNs every time even when we already got the successful NN.
That's because in RetryInvocationHandler.java, ProxyDescriptor's member proxyInfo is assigned only when it is constructed or when failover occurs. RequestHedgingProxyProvider.currentUsedProxy is null in both cases, so the only proxy we can get is always a dynamic proxy handled by RequestHedgingInvocationHandler.class. RequestHedgingInvocationHandler.class handles invoked method by calling multiple configured NNs.

Issue Links

• is related to: HDFS-12813 RequestHedgingProxyProvider can hide Exception thrown from the Namenode for proxy size of 1
• is related to: HDFS-7858 Improve HA Namenode Failover detection on the client