Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2915 Enable YARN RM scale out via federation using multiple RM's
  3. YARN-3666

Federation Intercepting and propagating AM- home RM communications

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      In order, to support transparent "spanning" of jobs across sub-clusters, all AM-RM communications are proxied (via YARN-2884).

      This JIRA tracks federation-specific mechanisms that decide how to "split/broadcast" requests to the RMs and "merge" answers to
      the AM.

      This the part one jira, which sets up the basic structure, without secondary subclusters. All requests are forwarded to home subcluster. Part two is in YARN-6511

      Attachments

        1. YARN-3666-YARN-2915.v9.patch
          37 kB
          Botong Huang
        2. YARN-3666-YARN-2915.v8.patch
          37 kB
          Botong Huang
        3. YARN-3666-YARN-2915.v7.patch
          37 kB
          Botong Huang
        4. YARN-3666-YARN-2915.v6.patch
          36 kB
          Botong Huang
        5. YARN-3666-YARN-2915.v5.patch
          46 kB
          Botong Huang
        6. YARN-3666-YARN-2915.v4.patch
          48 kB
          Botong Huang
        7. YARN-3666-YARN-2915.v3.patch
          46 kB
          Botong Huang
        8. YARN-3666-YARN-2915.v2.patch
          46 kB
          Botong Huang
        9. YARN-3666-YARN-2915.v1.patch
          47 kB
          Botong Huang

        Issue Links

          blocks

          Sub-task - The sub-task of the issue YARN-6511 Federation: transparently spanning application across multiple sub-clusters

          • Major - Major loss of function.
          • Resolved
          depends upon

          Sub-task - The sub-task of the issue YARN-3672 Create Facade for Federation State and Policy Store

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue YARN-3673 Create a FailoverProxy for Federation services

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue YARN-5390 Federation Subcluster Resolver

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue YARN-5531 UnmanagedAM pool manager for federating application across clusters

          • Major - Major loss of function.
          • Resolved
          is blocked by

          Sub-task - The sub-task of the issue YARN-2884 Proxying all AM-RM communications

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue YARN-6485 [Regression] TestFederationRMStateStoreService is failing with null pointer exception

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              botong Botong Huang
              kishorch Kishore Chaliparambil
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: