Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-1051

YARN Admission Control/Planner: enhancing the resource allocation model with time.

    XMLWordPrintableJSON

Details

    • Incompatible change, Reviewed

    Description

      In this umbrella JIRA we propose to extend the YARN RM to handle time explicitly, allowing users to "reserve" capacity over time. This is an important step towards SLAs, long-running services, workflows, and helps for gang scheduling.

      Attachments

        1. curino_MSR-TR-2013-108.pdf
          2.86 MB
          Carlo Curino
        2. socc14-paper15.pdf
          1.32 MB
          Carlo Curino
        3. techreport.pdf
          2.02 MB
          Subramaniam Krishnan
        4. YARN-1051.1.patch
          574 kB
          Subramaniam Krishnan
        5. YARN-1051.patch
          571 kB
          Subramaniam Krishnan
        6. YARN-1051-design.pdf
          600 kB
          Subramaniam Krishnan

        Issue Links

          incorporates

          Sub-task - The sub-task of the issue YARN-2386 Refactor common scheduler configurations into a base ResourceSchedulerConfig class

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue YARN-1707 Making the CapacityScheduler more dynamic

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue YARN-1708 Add a public API to reserve resources (part of YARN-1051)

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue YARN-1709 Admission Control: Reservation subsystem

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue YARN-1710 Admission Control: agents to allocate reservation

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue YARN-1711 CapacityOverTimePolicy: a policy to enforce quotas over time for YARN-1709

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue YARN-1712 Admission Control: plan follower

          • Major - Major loss of function.
          • Closed

          Sub-task - The sub-task of the issue YARN-2080 Admission Control: Integrate Reservation subsystem with ResourceManager

          • Major - Major loss of function.
          • Closed
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-14840 Tool to estimate resource requirements of an application pipeline based on prior executions

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-6548 Jobs executed can be configurated with specific users and time hours

          • Major - Major loss of function.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue YARN-2609 Example of use for the ReservationSystem

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-2572 Enhancements to the ReservationSytem/Planner

          • Major - Major loss of function.
          • Open

          Improvement - An improvement or enhancement to an existing feature or task. YARN-5326 Support for recurring reservations in the YARN ReservationSystem

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              curino Carlo Curino
              curino Carlo Curino
              Votes:
              0 Vote for this issue
              Watchers:
              70 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: