[HADOOP-18324] Interrupting RPC Client calls can lead to thread exhaustion

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 3.4.0, 2.10.2, 3.3.3
• Fix Version/s: 3.4.0, 3.3.5, 2.10.3
• Component/s: ipc
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.5, 2.10.3
• Hadoop Flags: Reviewed
• Assignee: Owen O'Malley
• Reporter: Owen O'Malley
• Created: 30/Jun/22 22:56
• Updated: 28/Jan/24 00:28
• Resolved: 18/Nov/22 16:51

Description

Currently the IPC client creates a boundless number of threads to write the rpc request to the socket. The NameNode uses timeouts on its RPC calls to the Journal Node and a stuck JN will cause the NN to create an infinite set of threads.

Issue Links

• causes: HADOOP-18624 Leaked calls may cause ObserverNameNode OOM.
• causes: HDFS-16853 The UT TestLeaseRecovery2#testHardLeaseRecoveryAfterNameNodeRestart failed because HADOOP-18324
• is caused by: HADOOP-6762 exception while doing RPC I/O closes channel
• links to: Github PR
• links to: GitHub Pull Request #4527
• links to: GitHub Pull Request #5369