[YARN-10754] RM Renew Delegation token thread should timeout and retry should also consider app new submitted.

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Assignee: Qi Zhu
• Reporter: Qi Zhu
• Created: 27/Apr/21 02:19
• Updated: 27/Apr/21 03:42

Description

As  YARN-9768 described:

Delegation token renewer thread in RM (DelegationTokenRenewer.java) renews HDFS tokens received to check for validity and expiration time.

This call is made to an underlying HDFS NN or Router Node (which has exact APIs as HDFS NN). If one of the nodes is bad and the renew call is stuck the thread remains stuck indefinitely. The thread should ideally timeout the renewToken and retry from the client's perspective.

But it only consider the app recovery, not consider the app submitted:

It will cause the app submitted not retry, when renew token (HDFS Namenode/ Router) timeout. 

Issue Links

• is related to: YARN-10722 Improvement to DelegationTokenRenewer in RM
• is related to: YARN-9768 RM Renew Delegation token thread should timeout and retry