Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8179

Preemption does not happen due to natural_termination_factor when DRF is used

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0, 3.1.1, 3.0.3
    • Component/s: None
    • Labels:
      None

      Description

      cluster

      • DominantResourceCalculator
      • QueueA : 50 (capacity) ~ 100 (max capacity)
      • QueueB : 50 (capacity) ~ 50 (max capacity)

      all of resources have been allocated to QueueA. (all Vcores are allocated to QueueA)
      if App1 is submitted to QueueB, over-utilized QueueA should be preempted.
      but, I’ve met the problem, which preemption does not happen. it caused that App1 AM can not allocated.

      when App1 is submitted, pending resources for asking App1 AM would be <Memory:2048, Vcores:1>
      so, Vcores which need to be preempted from QueueB should be 1.
      but, it can be 0 due to natural_termination_factor (default is 0.2)

      we should guarantee that resources not to be 0 even though applying natural_termination_factor

        Attachments

        1. YARN-8179.001.patch
          1 kB
          kyungwan nam
        2. YARN-8179.002.patch
          5 kB
          kyungwan nam
        3. YARN-8179.003.patch
          5 kB
          kyungwan nam

          Activity

            People

            • Assignee:
              kyungwan nam kyungwan nam
              Reporter:
              kyungwan nam kyungwan nam
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: