[YARN-3405] FairScheduler's preemption cannot happen between sibling in some case - ASF JIRA

XML

Word

Printable

JSON

Queue hierarchy described as below:

                  root
               /         \
       queue-1          queue-2       
          /      \
queue-1-1     queue-1-2

Assume cluster resource is 100

queue-1-1 and queue-2 has app. Each get 50 usage and 50 fairshare.
When queue-1-2 is active, and it cause some new preemption request for fairshare 25.
When preemption from root, it has possibility to find preemption candidate is queue-2. If so preemptContainerPreCheck for queue-2 return false because it's equal to its fairshare.
Finally queue-1-2 will be waiting for resource release form queue-1-1 itself.

What I expect here is that queue-1-2 preempt from queue-1-1.

is duplicated by

YARN-3414 FairScheduler's preemption may cause livelock

YARN-4134 FairScheduler preemption stops at queue level that all child queues are not over their fairshare

is part of

YARN-4752 FairScheduler should preempt for a ResourceRequest and all preempted containers should be on the same node

is related to

YARN-2154 FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request

YARN-6151 FS preemption does not consider child queues over fairshare if the parent is under

relates to

YARN-3453 Fair Scheduler: Parts of preemption logic uses DefaultResourceCalculator even in DRF mode causing thrashing

(1 relates to)