Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8250

Create another implementation of ContainerScheduler to support NM overallocation

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Won't Do
    • None
    • None
    • None
    • None

    Description

      YARN-6675 adds NM over-allocation support by modifying the existing ContainerScheduler and providing a utilizationBased resource tracker.

      However, the implementation adds a lot of complexity to ContainerScheduler, and future tweak of over-allocation strategy based on how much containers have been launched is even more complicated.

      As such, this Jira proposes a new ContainerScheduler that always launch guaranteed containers immediately and queues opportunistic containers. It relies on a periodical check to launch opportunistic containers. 

      Attachments

        1. YARN-8250-YARN-1011.00.patch
          148 kB
          Haibo Chen
        2. YARN-8250-YARN-1011.01.patch
          202 kB
          Haibo Chen
        3. YARN-8250-YARN-1011.02.patch
          202 kB
          Haibo Chen

        Issue Links

          supercedes

          Sub-task - The sub-task of the issue YARN-6675 Add NM support to launch opportunistic containers based on overallocation

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              haibochen Haibo Chen
              haibochen Haibo Chen
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: