Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6151

FS preemption does not consider child queues over fairshare if the parent is under

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0
    • Component/s: fairscheduler
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      This is preemption bug happens before 2.8.0, which also described in YARN-3405.

      Queue hierarchy described as below:

                        root
                     /         \
             queue-1          queue-2       
                /      \
      queue-1-1     queue-1-2
      

      Assume cluster resource is 100 and all queues have same weights.

      1. queue-1-1 and queue-2 has apps. Each get 50 usage and 50 fairshare.
      2. When queue-1-2 is active, supposedly it will preempt 25 from queue-1-1, but this doesn't happen because preemption happens top-down, queue-2 could be the preemption candidate as long as queue-2 is less needy than queue-1, and queue-2 doesn't exceed the fair share which means preemption won't happen.

      We need to filter out queue-2 since it isn't a valid candidate.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                yufeigu Yufei Gu
                Reporter:
                yufeigu Yufei Gu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: