[YARN-5353] ResourceManager can leak delegation tokens when they are shared across apps

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.0, 2.6.1
• Fix Version/s: 2.8.0, 2.6.5, 2.7.4, 3.0.0-alpha1
• Component/s: resourcemanager
• Target Version/s: 2.6.5, 2.7.4
• Hadoop Flags: Reviewed
• Assignee: Jason Darrell Lowe
• Reporter: Jason Darrell Lowe
• Created: 11/Jul/16 15:29
• Updated: 11/Oct/16 23:18
• Resolved: 13/Jul/16 02:30

Description

Recently saw a ResourceManager go into heavy GC. Heap dump showed that there were millions of delegation tokens on the heap. It looks like most of them belonged to the appTokens map in DelegationTokenRenewer. When an app completes and tokens are removed for it, I noticed that the appTokens entry for the app is not cleaned up if tokens were shared with other active apps.