Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-45 [Preemption] Scheduler feedback to AM to release containers
  3. YARN-4390

Do surgical preemption based on reserved container in CapacityScheduler

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.8.0, 2.7.3, 3.0.0-alpha1
    • 2.8.0, 3.0.0-alpha1
    • capacity scheduler
    • None
    • Reviewed

    Description

      There are multiple reasons why preemption could unnecessarily preempt containers. One is that an app could be requesting a large container (say 8-GB), and the preemption monitor could conceivably preempt multiple containers (say 8, 1-GB containers) in order to fill the large container request. These smaller containers would then be rejected by the requesting AM and potentially given right back to the preempted app.

      Attachments

        1. QueueNotHittingMax.jpg
          364 kB
          Eric Payne
        2. YARN-4390.1.patch
          224 kB
          Wangda Tan
        3. YARN-4390.2.patch
          225 kB
          Wangda Tan
        4. YARN-4390.3.branch-2.patch
          231 kB
          Wangda Tan
        5. YARN-4390.3.patch
          231 kB
          Wangda Tan
        6. YARN-4390.4.patch
          231 kB
          Wangda Tan
        7. YARN-4390.5.patch
          239 kB
          Wangda Tan
        8. YARN-4390.6.patch
          244 kB
          Wangda Tan
        9. YARN-4390.7.patch
          244 kB
          Wangda Tan
        10. YARN-4390.8.patch
          246 kB
          Wangda Tan
        11. YARN-4390.branch-2.8.001.patch
          244 kB
          Eric Payne
        12. YARN-4390.branch-2.8.002.patch
          244 kB
          Eric Payne
        13. YARN-4390.branch-2.8.003.patch
          245 kB
          Eric Payne
        14. YARN-4390-design.1.pdf
          106 kB
          Wangda Tan
        15. YARN-4390-test-results.pdf
          98 kB
          Wangda Tan

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue YARN-4822 Refactor existing Preemption Policy of CS for easier adding new approach to select preemption candidates

          • Major - Major loss of function.
          • Resolved
          duplicates

          Sub-task - The sub-task of the issue YARN-4108 CapacityScheduler: Improve preemption to only kill containers that would satisfy the incoming request

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              leftnoteasy Wangda Tan
              epayne Eric Payne
              Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: