Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-45

[Preemption] Scheduler feedback to AM to release containers

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.1.0-beta
    • Component/s: resourcemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The ResourceManager strikes a balance between cluster utilization and strict enforcement of resource invariants in the cluster. Individual allocations of containers must be reclaimed- or reserved- to restore the global invariants when cluster load shifts. In some cases, the ApplicationMaster can respond to fluctuations in resource availability without losing the work already completed by that task (MAPREDUCE-4584). Supplying it with this information would be helpful for overall cluster utilization [1]. To this end, we want to establish a protocol for the RM to ask the AM to release containers.

      [1] http://research.yahoo.com/files/yl-2012-003.pdf

        Attachments

        1. YARN-45_design_thoughts.pdf
          24 kB
          Carlo Curino
        2. YARN-45.1.patch
          48 kB
          Carlo Curino
        3. YARN-45.patch
          46 kB
          Carlo Curino
        4. YARN-45.patch
          16 kB
          Carlo Curino
        5. YARN-45.patch
          13 kB
          Carlo Curino
        6. YARN-45.patch
          14 kB
          Carlo Curino
        7. YARN-45.patch
          18 kB
          Carlo Curino
        8. YARN-45.patch
          15 kB
          Carlo Curino

          Issue Links

            Activity

              People

              • Assignee:
                curino Carlo Curino
                Reporter:
                chris.douglas Chris Douglas
              • Votes:
                0 Vote for this issue
                Watchers:
                30 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 1m
                  1m
                  Remaining:
                  Remaining Estimate - 1m
                  1m
                  Logged:
                  Time Spent - Not Specified
                  Not Specified