[HBASE-25832] Performance improvements for StochasticLoadBalancer

• Type: Umbrella
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Balancer, Performance
• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 01/May/21 00:52
• Updated: 10/Jul/21 22:53

Description

This is what I posted in the slack channel.

When refactoring balancer related code for moving them out to hbase-balancer module, I found that we will create a OffPeakHours instance in MoveCostFunction.cost every time. So I wrote a simple JMH test to test its performance.

https://github.com/Apache9/hbase-jmh-test/blob/master/src/main/java/org/apache/hadoop/hbase/master/balancer/MoveCostFunctionBenchmark.java

For the original version, the result is
Benchmark Mode Samples Score Score error Units
o.a.h.h.m.b.MoveCostFunctionBenchmark.computeCost avgt 10 242.312 3.853 ns/op
12:06
If I initialized OffPeakHours in constructor, then the result is
Benchmark Mode Samples Score Score error Units
o.a.h.h.m.b.MoveCostFunctionBenchmark.computeCost avgt 10 81.186 0.720 ns/op
12:06
If I also eliminate the conf.getFloat in the cost method(by storing them to a field member in constructor, the result is
Benchmark Mode Samples Score Score error Units
o.a.h.h.m.b.MoveCostFunctionBenchmark.computeCost avgt 10 4.419 0.035 ns/op
12:08
I think on a large cluster, the performance for balancer is a problem, so maybe we should try to avoid creating too many objects or calling conf.getXXX in the critical path? As we will call StochasticLoadBalancer.computeCost in every step when balancing cluster...

For large clusters, the performance of StochasticLoadBalancer will be very important, so I think we need to treat this problem seriously and try to improve.

Issue Links

• fixes: HBASE-22300 SLB doesn't perform well with increase in number of regions
• is related to: HBASE-25649 Complete the work on moving all the balancer related classes to hbase-balancer module

Sub-Tasks

• Use double instead of Double in StochasticLoadBalancer: Resolved
• Refactor and cleanup the code for CostFunction: Resolved
• Improve the performance for region load and region count related cost functions: Resolved
• Backport 'HBASE-25894 Improve the performance for region load and region count related cost functions' to branch-2.4 and branch-2.3: Resolved