[HDFS-12670] can't renew HDFS tokens with only the hdfs client jar

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.8.1
• Fix Version/s: 3.2.0, 3.1.1, 3.0.4
• Hadoop Flags: Reviewed
• Assignee: Arpit Agarwal
• Reporter: Thomas Graves
• Created: 16/Oct/17 22:15
• Updated: 23/May/19 23:14
• Resolved: 09/Jun/18 00:59

Description

It looks like the meta inf file for the hdfs token renewer is in the hadoop-hdfs.jar rather then the hadoop-hdfs-client.jar. This prevents a client from renewing a token because the service loader doesn't find the DFSClient Renewer class and falls back to use the TrivialRenewer.

You get an error like:
17/10/16 21:08:15 WARN Token: No TokenRenewer defined for token kind HDFS_DELEGATION_TOKEN
Exception in thread “main” java.lang.UnsupportedOperationException: Token renewal is not supported for HDFS_DELEGATION_TOKEN tokens
at org.apache.hadoop.security.token.Token$TrivialRenewer.renew(Token.java:497)

Issue Links

• blocks: SPARK-24493 Kerberos Ticket Renewal is failing in Hadoop 2.8+ and Hadoop 3