[HBASE-16958] Balancer recomputes block distributions every time balanceCluster() runs

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Balancer
• Assignee: Unassigned
• Reporter: Gary Helmling
• Created: 27/Oct/16 20:49
• Updated: 01/Jul/22 20:25
• Resolved: 27/Oct/16 23:58

Description

The change in HBASE-16570 modified the balancer to compute block distributions in parallel with a pool of 5 threads. However, because it does this every time Cluster is instantiated, it effectively bypasses the cache of block locations added in HBASE-14473:

In the LoadBalancer.balanceCluster() implementations (in StochasticLoadBalancer, SimpleLoadBalancer), we create a new Cluster instance.

In Cluster.<init>, we call registerRegion() on every HRegionInfo.

In registerRegion(), we do the following:

        regionLocationFutures.set(regionIndex,
            regionFinder.asyncGetBlockDistribution(region));

Then, back in Cluster.<init> we do a get() on each ListenableFuture in a loop.

So while we are doing the calls to get block locations in parallel with 5 threads, we're recomputing them every time balanceCluster() is called and not taking advantage of the cache at all.

Issue Links

• duplicates: HBASE-16570 Compute region locality in parallel at startup