Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5249

Oozie delegation token renewal fails for MR tokens in branch-1

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 1.1.2
    • Fix Version/s: None
    • Component/s: jobtracker, security
    • Labels:
      None

      Description

      When Oozie java action is executed, the following shows up in the job tracker log.
      013-05-14 21:51:45,643 ERROR org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: Exception renewing tokenIdent: 00 07 68 64 70 75 73 65 72 06 6d 61 70 72 65 64 26 6f 6f 7a 69 65 2f 63 6f 6e 64 6f 72 2d 73 65 63 2e 76 65 6e 6b 61 74 2e 6f 72 67 40 76 65 6e 6b 61 74 2e 6f 72 67 8a 01 3e a6 87 5e 5b 8a 01 3e ca 93 e2 5b 02 02, Kind: MAPREDUCE_DELEGATION_TOKEN, Service: 192.168.56.101:50300. Not rescheduled
      org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Client jt/condor-sec.venkat.org@venkat.org tries to renew a token with renewer specified as mapred
      at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:267)
      at org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:3878)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
      at org.apache.hadoop.ipc.Client.call(Client.java:1118)
      at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
      at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
      at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
      at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
      at org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:578)
      at org.apache.hadoop.security.token.Token.renew(Token.java:309)
      at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:221)
      at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:217)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
      at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run(DelegationTokenRenewal.java:216)
      at java.util.TimerThread.mainLoop(Timer.java:512)
      at java.util.TimerThread.run(Timer.java:462)

      Setting the renewer to Kerberos Local name does not help because AbstractDelegationTokenIdentifier sets the renewer to Kerberos shortname but JobTracker.renewDelegationToken uses the fullName. This essentially causes the renewal to fail

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                venkatnrangan Venkat Ranganathan
                Reporter:
                venkatnrangan Venkat Ranganathan
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: