[HBASE-14407] NotServingRegion: hbase region closed forever

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.98.10, 1.2.0, 1.1.2, 1.3.0
• Fix Version/s: 1.2.0, 1.3.0, 0.98.15, 1.0.3, 1.1.3
• Component/s: Region Assignment
• Hadoop Flags: Reviewed
• Assignee: Shuaifeng Zhou
• Reporter: Shuaifeng Zhou
• Created: 11/Sep/15 05:47
• Updated: 27/Jan/16 15:28
• Resolved: 26/Sep/15 01:20

Description

I found a situation may cause region closed forever, and this situation happend usually on my cluster, version is 0.98.10, but 1.1.2 also have the problem:
1, master send region open to regionserver
2, rs open a handler do openregion
3, rs return resopnse to master
3, master not received the response, or timeout, send open region again
4, rs already opened the region
5, master processAlreadyOpenedRegion, update regionstate open in master memory
6, master received zk message region opened(for some reason late, eg: net work), and triger update regionstate open, but find that region already opened, ERROR!
7, master send close region, and region be closed forever.