[HBASE-25130] [branch-1] Masters in-memory serverHoldings map is not cleared during hbck repair

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Fix Version/s: 1.7.1
• Component/s: hbck
• Assignee: Victor Li
• Reporter: Sandeep Guggilam
• Created: 01/Oct/20 00:20
• Updated: 09/Jul/21 20:45
• Resolved: 28/Jun/21 21:05

Description

Incase of repairing overlaps, hbck  essentially calls the closeRegion RPC on RS followed by offline RPC on Master to offline all the overlap regions that would be merged into a new region.

However the offline RPC doesn’t remove it from the serverHoldings map unless the new state is MERGED/SPLIT (https://github.com/apache/hbase/blob/branch-1/hbase-server/src/main/java/org/apache/hadoop/hbase/master/RegionStates.java#L719) but the new state in this case is OFFLINE.

This is actually intended to match with the META entries and would be removed later when the region is online on a different server. However, in our case , the region would never be online on a new server, hence the region info is never cleared from the map that is used by balancer and SCP for incorrect reeassignment.

We might need to tackle this by removing the entries from the map when hbck actually deletes the meta entries for this region which kind of matches the in-memory map’s expectation with the META state.

Issue Links

• links to: GitHub Pull Request #3402