Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3227

Timeline renew delegation token fails when RM user's TGT is expired

    Details

    • Target Version/s:

      Description

      When the RM user's kerberos TGT is expired, the RM renew delegation token operation fails as part of job submission. Expected behavior is that RM will relogin to get a new TGT.

      2015-02-06 18:54:05,617 DelegationTokenRenewer #25954 WARN
      security.DelegationTokenRenewer: Unable to add the application to the
      delegation token renewer.
      java.io.IOException: Failed to renew token: Kind: TIMELINE_DELEGATION_TOKEN,
      Service: timelineserver.example.com:4080, Ident: (owner=user,
      renewer=rmuser, realUser=oozie, issueDate=1423248845528,
      maxDate=1423853645528, sequenceNumber=9716, masterKeyId=9)
      at
      org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:443)
      at
      org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$800(DelegationTokenRenewer.java:77)
      at
      org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.handleDTRenewerAppSubmitEvent(DelegationTokenRenewer.java:808)
      at
      org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:789)
      at
      java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      at
      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      at java.lang.Thread.run(Thread.java:722)
      Caused by: java.io.IOException: HTTP status [401], message [Unauthorized]
      at
      org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:169)
      at
      org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.doDelegationTokenOperation(DelegationTokenAuthenticator.java:286)
      at
      org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.renewDelegationToken(DelegationTokenAuthenticator.java:211)
      at
      org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.renewDelegationToken(DelegationTokenAuthenticatedURL.java:414)
      at
      org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:374)
      at
      org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$2.run(TimelineClientImpl.java:360)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:415)
      at
      org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1694)
      at
      org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$4.run(TimelineClientImpl.java:429)
      at
      org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl$TimelineClientConnectionRetry.retryOn(TimelineClientImpl.java:161)
      at
      org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.operateDelegationToken(TimelineClientImpl.java:444)
      at
      org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.renewDelegationToken(TimelineClientImpl.java:378)
      at
      org.apache.hadoop.yarn.security.client.TimelineDelegationTokenIdentifier$Renewer.renew(TimelineDelegationTokenIdentifier.java:81)
      at org.apache.hadoop.security.token.Token.renew(Token.java:377)
      at
      org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$1.run(DelegationTokenRenewer.java:532)
      at
      org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$1.run(DelegationTokenRenewer.java:529)

        Attachments

        1. YARN-3227.test.patch
          2 kB
          Zhijie Shen
        2. YARN-3227.1.patch
          1 kB
          Zhijie Shen

          Activity

            People

            • Assignee:
              zjshen Zhijie Shen
              Reporter:
              jeagles Jonathan Eagles
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: