[MAPREDUCE-5364] Deadlock between RenewalTimerTask methods cancel() and run()

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.1
• Target Version/s: 1.2.1
• Hadoop Flags: Reviewed
• Assignee: Karthik Kambatla
• Reporter: Karthik Kambatla
• Created: 01/Jul/13 20:11
• Updated: 03/Nov/14 18:33
• Resolved: 11/Jul/13 20:29

Description

MAPREDUCE-4860 introduced a local variable cancelled in RenewalTimerTask to fix the race where DelegationTokenRenewal attempts to renew a token even after the job is removed. However, the patch also makes run() and cancel() synchronized methods leading to a potential deadlock against run()'s catch-block (error-path).

The deadlock stacks below:

 - org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.cancel() @bci=0, line=240 (Interpreted frame)
 - org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal.removeDelegationTokenRenewalForJob(org.apache.hadoop.mapreduce.JobID) @bci=109, line=319 (Interpreted frame)

 - org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal.removeFailedDelegationToken(org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$DelegationTokenToRenew) @bci=62, line=297 (Interpreted frame)
 - org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal.access$300(org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$DelegationTokenToRenew) @bci=1, line=47 (Interpreted frame)
 - org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run() @bci=148, line=234 (Interpreted frame)

Issue Links

• is related to: MAPREDUCE-5384 Races in DelegationTokenRenewal
• is related to: MAPREDUCE-4860 DelegationTokenRenewal attempts to renew token even after a job is removed