[HADOOP-10219] ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0, 2.6.0
• Fix Version/s: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2
• Component/s: ipc
• Assignee: Kihwal Lee
• Reporter: Steve Loughran
• Created: 10/Jan/14 15:22
• Updated: 05/Sep/18 05:29
• Resolved: 04/Sep/18 15:49

Description

When ClientCache.stopClient() is called to stop the IPC client, if the client
is blocked spinning due to a connectivity problem, it does not exit until the policy has timed out -so the stopClient() operation can hang for an extended period of time.

This can surface in the shutdown hook of FileSystem.cache.closeAll()

Also, Client.stop() is for used in NN switch from Standby to Active, and can therefore have very bad consequences and cause downtime.

Issue Links

• breaks: HDFS-5755 Filesystem close blocks SystemExit if a DFS client is trying to talk to a nonexistent NN
• is duplicated by: HADOOP-15683 Client.setupConnection should not block Client.stop() calls
• is related to: HADOOP-12464 Interrupted client may try to fail-over and retry
• is related to: HADOOP-12950 ShutdownHookManager should have a timeout for each of the Registered shutdown hook