Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8179

Preemption does not happen due to natural_termination_factor when DRF is used

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0, 3.1.1, 3.0.3, 2.10.1
    • None
    • None

    Description

      cluster

      • DominantResourceCalculator
      • QueueA : 50 (capacity) ~ 100 (max capacity)
      • QueueB : 50 (capacity) ~ 50 (max capacity)

      all of resources have been allocated to QueueA. (all Vcores are allocated to QueueA)
      if App1 is submitted to QueueB, over-utilized QueueA should be preempted.
      but, I’ve met the problem, which preemption does not happen. it caused that App1 AM can not allocated.

      when App1 is submitted, pending resources for asking App1 AM would be <Memory:2048, Vcores:1>
      so, Vcores which need to be preempted from QueueB should be 1.
      but, it can be 0 due to natural_termination_factor (default is 0.2)

      we should guarantee that resources not to be 0 even though applying natural_termination_factor

      Attachments

        1. YARN-8179.001.patch
          1 kB
          kyungwan nam
        2. YARN-8179.002.patch
          5 kB
          kyungwan nam
        3. YARN-8179.003.patch
          5 kB
          kyungwan nam

        Activity

          People

            kyungwan nam kyungwan nam
            kyungwan nam kyungwan nam
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: