[YARN-6793] Duplicated reservation in Fair Scheduler preemption

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Affects Version/s: 2.8.1, 3.0.0-alpha3
• Component/s: fairscheduler
• Assignee: Yufei Gu
• Reporter: Yufei Gu
• Created: 10/Jul/17 17:38
• Updated: 27/Jul/17 21:12
• Resolved: 27/Jul/17 21:12

Description

There is a delay between preemption happen and containers are killed. If resources released from nodes before container killing are not enough for the resource request preemption asking for, reservation happens again at that node.
E.g. scheduler reserves <memory 2048, vcore 2> in node 1 for app 1 while preemption. It will take 15s by default to kill containers in node 1 for fulfill that resource requests. If <memory 1024, vcore 1> was released from node 1 before the killing, scheduler reserves <memory 2048, vcore 2> again in node 1 for app1. The second reservation may never be unreserved.

Issue Links

• duplicates: YARN-6895 [FairScheduler] Preemption reservation may cause regular reservation leaks