[YARN-10589] Improve logic of multi-node allocation

• Type: Task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.0
• Assignee: Tanu Ajmera
• Reporter: Tanu Ajmera
• Created: 21/Jan/21 11:10
• Updated: 09/Mar/21 02:26

Description

for (String partititon : partitions) {
 if (current++ > start) {
 break;
 }
 CandidateNodeSet<FiCaSchedulerNode> candidates =
 cs.getCandidateNodeSet(partititon);
 if (candidates == null) {
 continue;
 }
 cs.allocateContainersToNode(candidates, false);
}

In above logic, if we have thousands of node in one partition, we will still repeatedly access all nodes of the partition thousands of times. There is no break point where if the partition is not same for the first node, it should stop checking other nodes in that partition.