[HBASE-21463] The checkOnlineRegionsReport can accidentally complete a TRSP

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.2.0
• Component/s: amv2
• Hadoop Flags: Reviewed
• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 09/Nov/18 14:35
• Updated: 10/Feb/19 23:14
• Resolved: 13/Nov/18 03:34

Description

On our testing cluster, we observe a race condition:
1. A regionServerReport request is built
2. A TRSP is scheduled to reopen the region
3. The region is closed at RS side
4. The OpenRegionProcedure is created
5. The regionServerReport generated at step 1 is executed, and we find that the region is opened on the RS, so we update the region state to OPEN.
6. The OpenRegionProcedure notices that the region has already been in the OPEN state so gives up and finishes.
7. The TRSP finishes.
8. The region is recorded as OPEN on the RS but actually not, and can not recover unless we use HBCK2.

Issue Links

• is related to: HBASE-21864 add region state version and reinstate YouAreDead exception in region report
• relates to: HBASE-21474 Backport "HBASE-21463 The checkOnlineRegionsReport can accidentally complete a TRSP" to branch-2.1 and branch-2.0
• links to: Review Board