Details
-
Sub-task
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
We should shuffle the nodes to avoid some nodes being preempted frequently.
Also, we should limit the num of nodes to make preemption more efficient.
Just like this,
// we should not iterate all nodes, that will be very slow long maxTryNodeNum = context.getPreemptionConfig().getToBePreemptedNodeMaxNumOnce(); if (potentialNodes.size() > maxTryNodeNum){ Collections.shuffle(potentialNodes); List<FSSchedulerNode> newPotentialNodes = new ArrayList<FSSchedulerNode>(); for (int i = 0; i < maxTryNodeNum; i++){ newPotentialNodes.add(potentialNodes.get(i)); } potentialNodes = newPotentialNodes;