-
Type:
Improvement
-
Status: Resolved
-
Priority:
Minor
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: 2.8.0, 3.0.0-alpha1
-
Component/s: fairscheduler
-
Labels:None
-
Hadoop Flags:Reviewed
See the attached picture of profiling result. The clone of Resource object within Resources.multiply() takes up *85%* (19.2 / 22.6) CPU time of the function FairScheduler.update().
The code of FSAppAttempt.updateDemand:
public void updateDemand() { demand = Resources.createResource(0); // Demand is current consumption plus outstanding requests Resources.addTo(demand, app.getCurrentConsumption()); // Add up outstanding resource requests synchronized (app) { for (Priority p : app.getPriorities()) { for (ResourceRequest r : app.getResourceRequests(p).values()) { Resource total = Resources.multiply(r.getCapability(), r.getNumContainers()); Resources.addTo(demand, total); } } } }
The code of Resources.multiply:
public static Resource multiply(Resource lhs, double by) { return multiplyTo(clone(lhs), by); }
The clone could be skipped by directly update the value of this.demand.