Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8250

Create another implementation of ContainerScheduler to support NM overallocation

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Do
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      YARN-6675 adds NM over-allocation support by modifying the existing ContainerScheduler and providing a utilizationBased resource tracker.

      However, the implementation adds a lot of complexity to ContainerScheduler, and future tweak of over-allocation strategy based on how much containers have been launched is even more complicated.

      As such, this Jira proposes a new ContainerScheduler that always launch guaranteed containers immediately and queues opportunistic containers. It relies on a periodical check to launch opportunistic containers. 

        Attachments

        1. YARN-8250-YARN-1011.00.patch
          148 kB
          Haibo Chen
        2. YARN-8250-YARN-1011.01.patch
          202 kB
          Haibo Chen
        3. YARN-8250-YARN-1011.02.patch
          202 kB
          Haibo Chen

          Issue Links

          supercedes

          Sub-task - The sub-task of the issue YARN-6675 Add NM support to launch opportunistic containers based on overallocation

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                haibochen Haibo Chen
                Reporter:
                haibochen Haibo Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: