[YARN-10794] Submitting jobs to a single subcluster will fail while AMRMProxy is enabled

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.2.1
• Component/s: nodemanager
• Assignee: Unassigned
• Reporter: Song Jiacheng
• Created: 31/May/21 02:56
• Updated: 23/Jun/21 07:33
• Resolved: 23/Jun/21 07:16

Description

Sorry for not knowing how to quote a issue...

https://issues.apache.org/jira/browse/YARN-9693

This issue has already raised this problem, but it seems that I can't submit job by the federation client while using the patch.

The original reason of this problem is that NM will set a local AMRMToken for AM if AMRMProxy is enabled, so that AM will fail if it contact with RM directly.

This problem makes it impossible to rolling upgrade to federation, cause we can't upgrade all the NMs and clients at one moment

So I developed another patch, using this patch I can submit jobs via the both ways.

My solution is that hold two tokens at the same time, and choose a right one during the building of RPC Client.

I tested this patch in some situations like AM recover, NM recover, no error found.

But still, I can't ensure this patch is good, so i wonder if there is a better solution.