[HADOOP-6498] IPC client bug may cause rpc call hang

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.18.3, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1
• Fix Version/s: 0.20.2
• Component/s: ipc
• Hadoop Flags: Reviewed
• Release Note: Correct synchronization error in IPC where handler thread could hang if request reader got an error.
• Assignee: Ruyue Ma
• Reporter: Ruyue Ma
• Created: 18/Jan/10 16:22
• Updated: 22/Jul/14 19:19
• Resolved: 26/Jan/10 23:05

Description

I can reproduce some rpc call hang bug when connection thread of ipc client receives response for outstanding call.

The stacks when hang occurs (TaskTracker):

Waiting on org.apache.hadoop.ipc.Client$Call@1c3cbb4b
Stack:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
org.apache.hadoop.ipc.Client.call(Client.java:691)
org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:216)
org.apache.hadoop.mapred.$Proxy4.heartbeat(Unknown Source)
org.apache.hadoop.mapred.TaskTracker.transmitHeartBeat(TaskTracker.java:1250)
org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:1082)
org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1785)
org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2796)

Issue Links

• duplicates: HADOOP-5760 Task process hanging on an RPC call