Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.8.0
-
None
-
Reviewed
Description
A number of issues have been reported with respect to preemption in FairScheduler along the lines of:
- FairScheduler preempts resources from nodes even if the resultant free resources cannot fit the incoming request.
- Preemption doesn't preempt from sibling queues
- Preemption doesn't preempt from sibling apps under the same queue that is over its fairshare
- ...
Filing this umbrella JIRA to group all the issues together and think of a comprehensive solution.
Attachments
Attachments
Issue Links
- breaks
-
YARN-6144 FairScheduler: preempted resources can become negative
- Resolved
- incorporates
-
YARN-4931 Preempted resources go back to the same application
- Open
-
YARN-2154 FairScheduler: Improve preemption to preempt only those containers that would satisfy the incoming request
- Resolved
-
YARN-3405 FairScheduler's preemption cannot happen between sibling in some case
- Resolved
-
YARN-3997 An Application requesting multiple core containers can't preempt running application made of single core containers
- Resolved
-
YARN-2457 FairScheduler: Handle preemption to help starved parent queues
- Resolved
-
YARN-3054 Preempt policy in FairScheduler may cause mapreduce job never finish
- Resolved
-
YARN-3121 FairScheduler preemption metrics
- Resolved
-
YARN-3902 Fair scheduler preempts ApplicationMaster
- Resolved
-
YARN-4120 FSAppAttempt.getResourceUsage() should not take preemptedResource into account
- Resolved
-
YARN-4133 Containers to be preempted leak in FairScheduler preemption logic.
- Resolved
-
YARN-4333 Fair scheduler should support preemption within a leaf queue
- Resolved
- is blocked by
-
YARN-5181 ClusterNodeTracker: add method to get list of nodes matching a specific resourceName
- Resolved
- is duplicated by
-
YARN-6076 Backport YARN-4752 (FS preemption changes) to branch-2
- Resolved
- is related to
-
YARN-8655 FairScheduler: FSStarvedApps is not thread safe
- Patch Available
YARN-6215adds a lock, and on the surface it appears would tackle this too. In any case, should we discuss this on a new JIRA so we can address any follow up work there?