[YARN-8127] Resource leak when async scheduling is enabled

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1, 2.10.2
• Hadoop Flags: Reviewed
• Assignee: Tao Yang
• Reporter: Weiwei Yang
• Created: 09/Apr/18 02:48
• Updated: 05/Oct/21 21:27
• Resolved: 05/Oct/21 21:27

Description

Brief steps to reproduce

1. Enable async scheduling, 5 threads
2. Submit a lot of jobs trying to exhaust cluster resource
3. After a while, observed NM allocated resource is more than resource requested by allocated containers

Looks like the commit phase is not sync handling reserved containers, causing some proposal incorrectly accepted, subsequently resource was deducted multiple times for a container.