[HBASE-3152] Disabling table stuck on regions-in-transition

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.90.0
• Component/s: master
• Assignee: Michael Stack
• Reporter: Sebastian Bauer
• Created: 26/Oct/10 08:22
• Updated: 12/Jun/22 00:30
• Resolved: 31/Mar/14 10:38

Description

after stress testing with only ICV and trying to disable table hbase run into regions-in-transition loop. I have logs and threads dumps. This region is on db2a regionserver, more logs as attachments

2010-10-26 09:32:00,838 INFO org.apache.hadoop.hbase.master.AssignmentManager: Waiting on NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792 to clear regions-in-transition
2010-10-26 09:32:01,838 INFO org.apache.hadoop.hbase.master.AssignmentManager: Waiting on NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792 to clear regions-in-transition
2010-10-26 09:32:02,838 INFO org.apache.hadoop.hbase.master.AssignmentManager: Waiting on NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792 to clear regions-in-transition
2010-10-26 09:32:02,842 INFO org.apache.hadoop.hbase.master.ServerManager: regionservers=2, averageload=1803
2010-10-26 09:32:03,104 INFO org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition timed out: NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792
2010-10-26 09:32:03,104 INFO org.apache.hadoop.hbase.master.AssignmentManager: Region has been PENDING_CLOSE or CLOSING for too long, running forced unassign again on region=NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0.
2010-10-26 09:32:03,104 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12be24df47b0000 Deleting existing unassigned node for 85b04e1097c676ef52b16d49305e0ab0 that is in expected state RS_ZK_REGION_CLOSING
2010-10-26 09:32:03,104 WARN org.apache.hadoop.hbase.zookeeper.ZKAssign: master:60000-0x12be24df47b0000 Attempting to delete unassigned node in RS_ZK_REGION_CLOSING state but node is in RS_ZK_REGION_CLOSED state
2010-10-26 09:32:03,838 INFO org.apache.hadoop.hbase.master.AssignmentManager: Waiting on NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792 to clear regions-in-transition
2010-10-26 09:32:04,838 INFO org.apache.hadoop.hbase.master.AssignmentManager: Waiting on NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792 to clear regions-in-transition
2010-10-26 09:32:05,838 INFO org.apache.hadoop.hbase.master.AssignmentManager: Waiting on NGolden_AC,,1287997478278.85b04e1097c676ef52b16d49305e0ab0. state=PENDING_CLOSE, ts=1288078080792 to clear regions-in-transition

Issue Links

• duplicates: HBASE-6537 Race between balancer and disable table can lead to inconsistent cluster