[YARN-1290] Let continuous scheduling achieve more balanced task assignment

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.3.0
• Assignee: Wei Yan
• Reporter: Wei Yan
• Created: 10/Oct/13 20:19
• Updated: 24/Feb/14 20:57
• Resolved: 31/Oct/13 22:07

Description

Currently, in continuous scheduling (YARN-1010), in each round, the thread iterates over pre-ordered nodes and assigns tasks. This mechanism may overload the first several nodes, while the latter nodes have no tasks.

We should sort all nodes according to available resource. In each round, always assign tasks to nodes with larger capacity, which can balance the load distribution among all nodes.