[MAPREDUCE-5249] Oozie delegation token renewal fails for MR tokens in branch-1

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.1.2
• Component/s: jobtracker, security
• Assignee: Venkat Ranganathan
• Reporter: Venkat Ranganathan
• Created: 15/May/13 05:36
• Updated: 04/Jul/13 22:58
• Resolved: 04/Jul/13 22:55

Description

When Oozie java action is executed, the following shows up in the job tracker log.
013-05-14 21:51:45,643 ERROR org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal: Exception renewing tokenIdent: 00 07 68 64 70 75 73 65 72 06 6d 61 70 72 65 64 26 6f 6f 7a 69 65 2f 63 6f 6e 64 6f 72 2d 73 65 63 2e 76 65 6e 6b 61 74 2e 6f 72 67 40 76 65 6e 6b 61 74 2e 6f 72 67 8a 01 3e a6 87 5e 5b 8a 01 3e ca 93 e2 5b 02 02, Kind: MAPREDUCE_DELEGATION_TOKEN, Service: 192.168.56.101:50300. Not rescheduled
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Client jt/[e-mail] tries to renew a token with renewer specified as mapred
at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:267)
at org.apache.hadoop.mapred.JobTracker.renewDelegationToken(JobTracker.java:3878)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
at org.apache.hadoop.ipc.Client.call(Client.java:1118)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at org.apache.hadoop.mapred.$Proxy8.renewDelegationToken(Unknown Source)
at org.apache.hadoop.mapred.JobClient$Renewer.renew(JobClient.java:578)
at org.apache.hadoop.security.token.Token.renew(Token.java:309)
at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:221)
at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask$1.run(DelegationTokenRenewal.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
at org.apache.hadoop.mapreduce.security.token.DelegationTokenRenewal$RenewalTimerTask.run(DelegationTokenRenewal.java:216)
at java.util.TimerThread.mainLoop(Timer.java:512)
at java.util.TimerThread.run(Timer.java:462)

Setting the renewer to Kerberos Local name does not help because AbstractDelegationTokenIdentifier sets the renewer to Kerberos shortname but JobTracker.renewDelegationToken uses the fullName. This essentially causes the renewal to fail

Issue Links

• duplicates: MAPREDUCE-5375 Delegation Token renewal exception in jobtracker logs