[YARN-1010] FairScheduler: decouple container scheduling from nodemanager heartbeats

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.1.0-beta
• Fix Version/s: 2.3.0
• Component/s: scheduler
• Hadoop Flags: Reviewed
• Assignee: Wei Yan
• Reporter: Alejandro Abdelnur
• Created: 01/Aug/13 16:20
• Updated: 08/Dec/17 14:35
• Resolved: 01/Oct/13 19:57

Description

Currently scheduling for a node is done when a node heartbeats.

For large cluster where the heartbeat interval is set to several seconds this delays scheduling of incoming allocations significantly.

We could have a continuous loop scanning all nodes and doing scheduling. If there is availability AMs will get the allocation in the next heartbeat after the one that placed the request.

Issue Links

• is related to: YARN-6487 FairScheduler: remove continuous scheduling (YARN-1010)