[YARN-8631] YARN RM fails to add the application to the delegation token renewer on recovery

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 3.1.0
• Component/s: yarn
• Assignee: Umesh Mittal
• Reporter: Sanjay Divgi
• Created: 07/Aug/18 10:18
• Updated: 01/Apr/21 06:32

Description

On HA cluster we have observed that yarn resource manager fails to add the application to the delegation token renewer on recovery.

Below is the error:

2018-08-07 08:41:23,850 INFO security.DelegationTokenRenewer (DelegationTokenRenewer.java:renewToken(635)) - Renewed delegation-token= [Kind: TIMELINE_DELEGATION_TOKEN, Service: 172.27.84.192:8188, Ident: (TIMELINE_DELEGATION_TOKEN owner=hrt_qa_hive_spark, renewer=yarn, realUser=, issueDate=1533624642302, maxDate=1534229442302, sequenceNumber=18, masterKeyId=4);exp=1533717683478; apps=[application_1533623972681_0001]]
2018-08-07 08:41:23,855 WARN security.DelegationTokenRenewer (DelegationTokenRenewer.java:handleDTRenewerAppRecoverEvent(955)) - Unable to add the application to the delegation token renewer on recovery.
java.lang.NullPointerException
at org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleAppSubmitEvent(DelegationTokenRenewer.java:522)
at org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.handleDTRenewerAppRecoverEvent(DelegationTokenRenewer.java:953)
at org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer.access$700(DelegationTokenRenewer.java:79)
at org.apache.hadoop.yarn.server.resourcemanager.security.DelegationTokenRenewer$DelegationTokenRenewerRunnable.run(DelegationTokenRenewer.java:912)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Issue Links

• is related to: YARN-7962 Race Condition When Stopping DelegationTokenRenewer causes RM crash during failover