[YARN-9708] Yarn Router Support DelegationToken

• Type: New Feature
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: router
• Labels: pull-request-available
• Target Version/s: 3.5.0
• Flags: Patch
• Assignee: Shilun Fan
• Reporter: Xie YiFan
• Created: 26/Jul/19 09:18
• Updated: 04/Jan/24 08:36

Description

1.we use router as proxy to manage multiple cluster which be independent of each other in order to apply unified client. Thus, we implement our customized AMRMProxyPolicy that doesn't broadcast ResourceRequest to other cluster.

2.Our production environment need kerberos. But router doesn't support SecureLogin for now.
https://issues.apache.org/jira/browse/YARN-6539 desn't work. So we improvement it.

3.Some framework like oozie would get Token via yarnclient#getDelegationToken which router doesn't support. Our solution is that adding homeCluster to ApplicationSubmissionContextProto & GetDelegationTokenRequestProto. Job would be submitted with specified clusterid so that router knows which cluster to submit this job. Router would get Token from one RM according to specified clusterid when client call getDelegation meanwhile apply some mechanism to save this token in memory.

 

Issue Links

• relates to: YARN-5597 YARN Federation improvements
• links to: GitHub Pull Request #4740
• links to: GitHub Pull Request #4746

Sub-Tasks

• [Federation] Router Support DelegationToken storeNewMasterKey/removeStoredMasterKey With MemoryStateStore: Resolved
• [Federation] Router Support DelegationToken storeNewToken/updateStoredToken/removeStoredToken With MemoryStateStore: Resolved
• [Federation] Router Support DelegationToken in MemoryStore mode: Resolved
• [Federation] YARN Federation Router Supports Cross-Origin.: Resolved
• [Federation] Router Support DelegationToken With SQL: Resolved
• [Federation] Router Support DelegationToken With ZK: Resolved
• [Federation] Make Router Delegation token secret manager completely stateless: Open