Hadoop YARN
  1. Hadoop YARN
  2. YARN-47 [Umbrella] Security issues in YARN
  3. YARN-503

DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

    Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.3, 3.0.0, 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      The first Job/App to register a token is the one which DelegationTokenRenewer associates with a a specific Token. An attempt to remove/cancel these shared tokens by subsequent jobs doesn't work - since the JobId will not match.
      As a result, Even if subsequent jobs have MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be cancelled when those jobs complete.
      Tokens will eventually be removed from the RM / JT when the service that issued them considers them to have expired or via an explicit cancelDelegationTokens call (not implemented yet in 23).
      A side affect of this is that the same delegation token will end up being renewed multiple times (a separate TimerTask for each job which uses the token).

      DelegationTokenRenewer could maintain a reference count/list of jobIds for shared tokens.

      1. YARN-503.patch
        34 kB
        Daryn Sharp
      2. YARN-503.patch
        34 kB
        Daryn Sharp

        Issue Links

          Activity

            People

            • Assignee:
              Daryn Sharp
              Reporter:
              Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development