Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-45

[Preemption] Scheduler feedback to AM to release containers

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0-beta
    • resourcemanager
    • None
    • Reviewed

    Description

      The ResourceManager strikes a balance between cluster utilization and strict enforcement of resource invariants in the cluster. Individual allocations of containers must be reclaimed- or reserved- to restore the global invariants when cluster load shifts. In some cases, the ApplicationMaster can respond to fluctuations in resource availability without losing the work already completed by that task (MAPREDUCE-4584). Supplying it with this information would be helpful for overall cluster utilization [1]. To this end, we want to establish a protocol for the RM to ask the AM to release containers.

      [1] http://research.yahoo.com/files/yl-2012-003.pdf

      Attachments

        1. YARN-45.patch
          15 kB
          Carlo Curino
        2. YARN-45.patch
          18 kB
          Carlo Curino
        3. YARN-45.patch
          14 kB
          Carlo Curino
        4. YARN-45.patch
          13 kB
          Carlo Curino
        5. YARN-45.patch
          16 kB
          Carlo Curino
        6. YARN-45.patch
          46 kB
          Carlo Curino
        7. YARN-45.1.patch
          48 kB
          Carlo Curino
        8. YARN-45_design_thoughts.pdf
          24 kB
          Carlo Curino

        Issue Links

          Activity

            People

              curino Carlo Curino
              cdouglas Christopher Douglas
              Votes:
              0 Vote for this issue
              Watchers:
              30 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1m
                  1m
                  Remaining:
                  Remaining Estimate - 1m
                  1m
                  Logged:
                  Time Spent - Not Specified
                  Not Specified