[YARN-6355] [Umbrella] Preprocessor framework for AM and Client interactions with the RM

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: amrmproxy resourcemanager
• Assignee: Arun Suresh
• Reporter: Arun Suresh
• Created: 16/Mar/17 21:15
• Updated: 08/Jan/19 19:02

Description

Currently on the NM, we have the AMRMProxy framework to intercept the AM <-> RM communication and enforce policies. This is used both by YARN federation (YARN-2915) as well as Distributed Scheduling (YARN-2877).

This JIRA proposes to introduce a similar framework on the the RM side, so that pluggable policies can be enforced on ApplicationMasterService centrally as well.

This would be similar in spirit to a Java Servlet Filter Chain. Where the order of the interceptors can declared externally.

Once possible usecase would be:
the OpportunisticContainerAllocatorAMService is implemented as a wrapper over the ApplicationMasterService. It would probably be better to implement it as an Interceptor.

Issue Links

• is depended upon by: YARN-1547 Prevent DoS of ApplicationMasterProtocol by putting in limits
• is related to: YARN-2884 Proxying all AM-RM communications
• is related to: YARN-6777 Support for ApplicationMasterService processing chain of interceptors

Sub-Tasks

• Refactor ApplicaitonMasterService to move actual processing logic to a separate class: Resolved
• Support for ApplicationMasterService processing chain of interceptors: Resolved
• Move Router ClientRMServices Interceptor and chain into yarn api and common package: Open