[HBASE-27302] Adding a trigger for Stochastica Balancer to safeguard for upper bound outliers.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Balancer
• Assignee: Unassigned Assign to me
• Reporter: Clara Xiong
• Created: 14/Aug/22 01:02
• Updated: 14/Aug/22 03:20

Description

In large clusters, if one outlier has a lot of regions, the calculated imbalance for  RegionCountSkewCostFunction is quite low and often fails to trigger the balancer.

For example, a node with twice average count on a 400-node cluster only produce an imbalance of 0.004 < 0.02(current default threshold to trigger balancer). An empty node also have similar effect but we have a safeguard in place. https://issues.apache.org/jira/browse/HBASE-24139

We can add a safeguard for this so  we don't have to lower threshold on larger clusters that makes the balancer more sensitive to other minor imbalances.

Issue Links

• is a child of: HBASE-25697 StochasticBalancer improvement for large scale clusters
• is related to: HBASE-25625 StochasticBalancer CostFunctions needs a better way to evaluate region count distribution
• is related to: HBASE-24139 Balancer should avoid leaving idle region servers