Details
-
Sub-task
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
We have a couple of scenarios that we force balancing:
- idle servers
- co-hosted regions
The code path quit before populating the sum of multiplier of cost functions. This causes wrong value reported in logging. As below, the weighted average is not divide by total weight. This causes inconsistent log among iterations.
2021-09-24 21:46:57,881 INFO org.apache.hadoop.hbase.master.balancer.S*tocha*sticLoadBalancer: Running balancer because at least one server hosts replicas of the same region.
2021-09-24 21:46:57,881 INFO org.apache.hadoop.hbase.master.balancer.S*tocha*sticLoadBalancer: Start S*tocha*sticLoadBalancer.balancer, initial weighted average imbalance=6389.260497305375, functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.06659036267913739); PrimaryRegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.05296760285663541); MoveCostFunction : (multiplier=7.0, imbalance=0.0, balanced); ServerLocalityCostFunction : (multiplier=25.0, imbalance=0.46286750487559114); RackLocalityCostFunction : (multiplier=15.0, imbalance=0.2569525347374165); TableSkewCostFunction : (multiplier=500.0, imbalance=0.3760689783169534); RegionReplicaHostCostFunction : (multiplier=100000.0, imbalance=0.0553889913899139); RegionReplicaRackCostFunction : (multiplier=10000.0, imbalance=0.05854089790897909); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.06969346106898068); WriteRequestCostFunction : (multiplier=5.0, imbalance=0.07834116112410174); MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.12533769793201735); StoreFileCostFunction : (multiplier=5.0, imbalance=0.06921401085082914); computedMaxSteps=5577401600
Attachments
Issue Links
- is a child of
-
HBASE-25697 StochasticBalancer improvement for large scale clusters
-
- Open
-
- links to