[YARN-10229] [Federation] Client should be able to submit application to RM directly using normal client conf

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.1
• Fix Version/s: 3.4.0, 3.3.1
• Component/s: amrmproxy, federation
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Flags: Patch
• Assignee: Bilwa S T
• Reporter: JohnsonGuo
• Created: 09/Apr/20 11:44
• Updated: 16/Sep/23 02:00
• Resolved: 03/Aug/20 07:28

Description

Scenario： When enable the yarn federation feature with multi yarn clusters, one can submit their job to yarn-router by modified their client configuration with yarn router address.

But if one still wants to submit their jobs via the original client (before enable federation) to RM directly, it will encounter the AMRMToken exception.  That means once enable federation ,if some one want to submit job, they have to  modify the client conf.

 

one possible solution for this Scenario is:

In NodeManger, when the client ApplicationMaster request comes:

• get the client job.xml  from HDFS "".
• parse the "yarn.resourcemanager.scheduler.address" parameter in job.xml
• if the value of the parameter is "localhost:8049"(AMRM address),then do the AMRMToken valid process
• if the value of the parameter is "rm:port"(rm address),then skip the AMRMToken valid process

 

 

 

Issue Links

• relates to: YARN-5597 YARN Federation improvements