Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.23.3, 2.0.0-alpha, 3.0.0-alpha1
-
None
-
None
Description
The first Job/App to register a token is the one which DelegationTokenRenewer associates with a a specific Token. An attempt to remove/cancel these shared tokens by subsequent jobs doesn't work - since the JobId will not match.
As a result, Even if subsequent jobs have MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be cancelled when those jobs complete.
Tokens will eventually be removed from the RM / JT when the service that issued them considers them to have expired or via an explicit cancelDelegationTokens call (not implemented yet in 23).
A side affect of this is that the same delegation token will end up being renewed multiple times (a separate TimerTask for each job which uses the token).
DelegationTokenRenewer could maintain a reference count/list of jobIds for shared tokens.
Attachments
Attachments
Issue Links
- is part of
-
MAPREDUCE-3101 [Umbrella] Security issues in YARN
- Resolved
- is required by
-
MAPREDUCE-5096 Deprecate ability to not cancel tokens when job completes
- Open