Affects Version/s: 0.23.3, 3.0.0, 2.0.0-alpha
Fix Version/s: None
The first Job/App to register a token is the one which DelegationTokenRenewer associates with a a specific Token. An attempt to remove/cancel these shared tokens by subsequent jobs doesn't work - since the JobId will not match.
As a result, Even if subsequent jobs have MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be cancelled when those jobs complete.
Tokens will eventually be removed from the RM / JT when the service that issued them considers them to have expired or via an explicit cancelDelegationTokens call (not implemented yet in 23).
A side affect of this is that the same delegation token will end up being renewed multiple times (a separate TimerTask for each job which uses the token).
DelegationTokenRenewer could maintain a reference count/list of jobIds for shared tokens.
|Status||Patch Available [ 10002 ]||Open [ 1 ]|
|Status||Open [ 1 ]||Patch Available [ 10002 ]|
|Project||Hadoop Map/Reduce [ 12310941 ]||Hadoop YARN [ 12313722 ]|
|Affects Version/s||2.0.0-alpha [ 12323271 ]|
|Affects Version/s||3.0.0 [ 12323268 ]|
|Affects Version/s||0.23.3 [ 12322841 ]|
|Affects Version/s||0.23.0 [ 12315570 ]|
|Affects Version/s||1.0.0 [ 12318240 ]|
|Component/s||resourcemanager [ 12319322 ]|
|Component/s||mrv2 [ 12314301 ]|
|Component/s||resourcemanager [ 12315340 ]|
|Assignee||Daryn Sharp [ daryn ]|