Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-45 [Preemption] Scheduler feedback to AM to release containers
  3. YARN-568

FairScheduler: support for work-preserving preemption

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.1.0-beta
    • scheduler
    • None
    • Reviewed

    Description

      In the attached patch, we modified the FairScheduler to substitute its preemption-by-killling with a work-preserving version of preemption (followed by killing if the AMs do not respond quickly enough). This should allows to run preemption checking more often, but kill less often (proper tuning to be investigated). Depends on YARN-567 and YARN-45, is related to YARN-569.

      Attachments

        1. YARN-568-2.patch
          23 kB
          Sandy Ryza
        2. YARN-568-2.patch
          23 kB
          Sandy Ryza
        3. YARN-568-1.patch
          19 kB
          Christopher Douglas
        4. YARN-568.patch
          12 kB
          Carlo Curino
        5. YARN-568.patch
          11 kB
          Carlo Curino

        Issue Links

          incorporates

          Sub-task - The sub-task of the issue YARN-567 RM changes to support preemption for FairScheduler and CapacityScheduler

          • Major - Major loss of function.
          • Closed
          is blocked by

          Sub-task - The sub-task of the issue YARN-567 RM changes to support preemption for FairScheduler and CapacityScheduler

          • Major - Major loss of function.
          • Closed
          Is contained by

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-4584 Umbrella: Preemption and restart of MapReduce tasks

          • Major - Major loss of function.
          • Open
          is related to

          Sub-task - The sub-task of the issue YARN-569 CapacityScheduler: support for preemption (using a capacity monitor)

          • Major - Major loss of function.
          • Closed
          relates to

          Sub-task - The sub-task of the issue MAPREDUCE-5189 Basic AM changes to support preemption requests (per YARN-45)

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue MAPREDUCE-5176 Preemptable annotations (to support preemption in MR)

          • Major - Major loss of function.
          • Closed

          Activity

            People

              curino Carlo Curino
              curino Carlo Curino
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: