[HDFS-4646] createNNProxyWithClientProtocol ignores configured timeout value

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.3-alpha, 2.0.4-alpha, 3.0.0-alpha1
• Fix Version/s: 2.0.4-alpha
• Component/s: namenode

Environment

Linux

• Assignee: Jagane Sundar
• Reporter: Jagane Sundar
• Created: 28/Mar/13 18:48
• Updated: 12/May/16 18:16
• Resolved: 05/Apr/13 20:50

Description

The Client RPC I/O timeout mechanism appears to be configured by two core-site.xml paramters:

1. A boolean ipc.client.ping
2. A numeric value ipc.ping.interval

If ipc.client.ping is true, then we send a RPC ping every ipc.ping.interval milliseconds
If ipc.client.ping is false, then ipc.ping.interval turns into the socket timeout value.

The bug here is that while creating a Non HA proxy, the configured timeout value is ignored, and 0 is passed in. 0 is taken to mean 'wait forever' and the client RPC socket never times out.

Note that this bug is reproducible only in the case where the NN machine dies, i.e. the TCP stack with the NN IP address stops responding completely. The code does not take this path when you do a 'kill -9' of the NN process, since there is a TCP stack that is alive and sends out a TCP RST to the client, and that results in a socket error (not a timeout).

The fix is to pass in the correct configured value for timeout by calling Client.getTimeout(conf) instead of passing in 0.