[HDFS-5322] HDFS delegation token not found in cache errors seen on secure HA clusters

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.1-beta
• Fix Version/s: 2.3.0
• Component/s: ha
• Hadoop Flags: Reviewed
• Assignee: Jing Zhao
• Reporter: Arpit Gupta
• Created: 07/Oct/13 22:52
• Updated: 27/Feb/15 18:46
• Resolved: 11/Oct/13 21:58

Description

While running HA tests we have seen issues were we see HDFS delegation token not found in cache errors causing jobs running to fail.

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157)
|2013-10-06 20:14:51,193 INFO  [main] mapreduce.Job: Task Id : attempt_1381090351344_0001_m_000007_0, Status : FAILED
Error: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken): token (HDFS_DELEGATION_TOKEN token 11 for hrt_qa) can't be found in cache
at org.apache.hadoop.ipc.Client.call(Client.java:1347)
at org.apache.hadoop.ipc.Client.call(Client.java:1300)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy10.getBlockLocations(Unknown Source)

Issue Links

• is depended upon by: HADOOP-11692 Improve authentication failure WARN message to avoid user confusion