Hadoop YARN
  1. Hadoop YARN
  2. YARN-47 [Umbrella] Security issues in YARN
  3. YARN-503

DelegationTokens will be renewed forever if multiple jobs share tokens and the first one sets JOB_CANCEL_DELEGATION_TOKEN to false

    Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.23.3, 3.0.0, 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: resourcemanager
    • Labels:
      None

      Description

      The first Job/App to register a token is the one which DelegationTokenRenewer associates with a a specific Token. An attempt to remove/cancel these shared tokens by subsequent jobs doesn't work - since the JobId will not match.
      As a result, Even if subsequent jobs have MRJobConfig.JOB_CANCEL_DELEGATION_TOKEN set to true - tokens will not be cancelled when those jobs complete.
      Tokens will eventually be removed from the RM / JT when the service that issued them considers them to have expired or via an explicit cancelDelegationTokens call (not implemented yet in 23).
      A side affect of this is that the same delegation token will end up being renewed multiple times (a separate TimerTask for each job which uses the token).

      DelegationTokenRenewer could maintain a reference count/list of jobIds for shared tokens.

      1. YARN-503.patch
        34 kB
        Daryn Sharp
      2. YARN-503.patch
        34 kB
        Daryn Sharp

        Issue Links

          Activity

          Siddharth Seth created issue -
          Eugene Koontz made changes -
          Field Original Value New Value
          Link This issue is part of MAPREDUCE-3101 [ MAPREDUCE-3101 ]
          Daryn Sharp made changes -
          Assignee Daryn Sharp [ daryn ]
          Daryn Sharp made changes -
          Project Hadoop Map/Reduce [ 12310941 ] Hadoop YARN [ 12313722 ]
          Key MAPREDUCE-3979 YARN-503
          Affects Version/s 2.0.0-alpha [ 12323271 ]
          Affects Version/s 3.0.0 [ 12323268 ]
          Affects Version/s 0.23.3 [ 12322841 ]
          Affects Version/s 0.23.0 [ 12315570 ]
          Affects Version/s 1.0.0 [ 12318240 ]
          Component/s resourcemanager [ 12319322 ]
          Component/s mrv2 [ 12314301 ]
          Component/s resourcemanager [ 12315340 ]
          Daryn Sharp made changes -
          Link This issue is required by MAPREDUCE-5096 [ MAPREDUCE-5096 ]
          Daryn Sharp made changes -
          Attachment YARN-503.patch [ 12575120 ]
          Daryn Sharp made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Daryn Sharp made changes -
          Attachment YARN-503.patch [ 12577915 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Vinod Kumar Vavilapalli made changes -
          Parent YARN-47 [ 12605130 ]
          Issue Type Bug [ 1 ] Sub-task [ 7 ]

            People

            • Assignee:
              Daryn Sharp
              Reporter:
              Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:

                Development