Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6355

[Umbrella] Preprocessor framework for AM and Client interactions with the RM

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None

      Description

      Currently on the NM, we have the AMRMProxy framework to intercept the AM <-> RM communication and enforce policies. This is used both by YARN federation (YARN-2915) as well as Distributed Scheduling (YARN-2877).

      This JIRA proposes to introduce a similar framework on the the RM side, so that pluggable policies can be enforced on ApplicationMasterService centrally as well.

      This would be similar in spirit to a Java Servlet Filter Chain. Where the order of the interceptors can declared externally.

      Once possible usecase would be:
      the OpportunisticContainerAllocatorAMService is implemented as a wrapper over the ApplicationMasterService. It would probably be better to implement it as an Interceptor.

        Attachments

        1. YARN-6355.007.patch
          90 kB
          Arun Suresh
        2. YARN-6355.006.patch
          90 kB
          Arun Suresh
        3. YARN-6355.005.patch
          85 kB
          Arun Suresh
        4. YARN-6355.004.patch
          91 kB
          Arun Suresh
        5. YARN-6355.003.patch
          92 kB
          Arun Suresh
        6. YARN-6355-one-pager.pdf
          235 kB
          Arun Suresh
        7. YARN-6355.002.patch
          91 kB
          Arun Suresh
        8. YARN-6355.001.patch
          67 kB
          Arun Suresh

          Issue Links

            Activity

              People

              • Assignee:
                asuresh Arun Suresh
                Reporter:
                asuresh Arun Suresh
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: