[HBASE-23173] ArrayIndexOutOfBoundsException in BaseLoadBalancer$Cluster.removeRegion

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.1.1
• Assignee: Unassigned
• Reporter: Michael Stack
• Created: 14/Oct/19 23:48
• Updated: 11/Nov/19 17:23
• Resolved: 15/Oct/19 04:48

Description

175 nodes with 12237 regions.

2019-10-14 23:45:47,823 INFO org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer: start StochasticLoadBalancer.balancer, initCost=110946.81226093257, functionCost=RegionCountSkewCostFunction : (500.0, 0.9166818227745739); PrimaryRegionCountSkewCostFunction : (500.0, 0.9166048407040664); MoveCostFunction : (7.0, 0.0); ServerLocalityCostFunction : (25.0, 0.5597851965798261); RackLocalityCostFunction : (15.0, 0.5811675989545179); TableSkewCostFunction : (35.0, 0.08287855195593785); RegionReplicaHostCostFunction : (100000.0, 1.0); RegionReplicaRackCostFunction : (10000.0, 1.0); ReadRequestCostFunction : (5.0, 0.0); WriteRequestCostFunction : (5.0, 0.0); MemStoreSizeCostFunction : (5.0, 0.0); StoreFileCostFunction : (5.0, 0.9112071951944016);  computedMaxSteps: 1000000
2019-10-14 23:45:47,933 ERROR org.apache.hadoop.hbase.ScheduledChore: Caught error
java.lang.ArrayIndexOutOfBoundsException: Index 145 out of bounds for length 145
	at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.removeRegion(BaseLoadBalancer.java:873)
	at org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer$Cluster.doAction(BaseLoadBalancer.java:716)
	at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:406)
	at org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer.balanceCluster(StochasticLoadBalancer.java:317)
	at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1663)
	at org.apache.hadoop.hbase.master.HMaster.balance(HMaster.java:1580)
	at org.apache.hadoop.hbase.master.balancer.BalancerChore.chore(BalancerChore.java:49)
	at org.apache.hadoop.hbase.ScheduledChore.run(ScheduledChore.java:186)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
	at org.apache.hadoop.hbase.JitterScheduledThreadPoolExecutorImpl$JitteredRunnableScheduledFuture.run(JitterScheduledThreadPoolExecutorImpl.java:111)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Issue Links

• duplicates: HBASE-22739 ArrayIndexOutOfBoundsException when balance
• links to: GitHub Pull Request #723