[HBASE-2575] Fault scenario of dead root drive on RS causes cluster lockup

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Cannot Reproduce
• Affects Version/s: 0.90.0
• Component/s: regionserver
• Assignee: Unassigned
• Reporter: Todd Lipcon
• Created: 19/May/10 23:29
• Updated: 11/Jun/22 23:34
• Resolved: 16/Jul/14 22:04

Description

We performed a fault test where we physically pulled the root drive out of a machine while it was on. The regionserver continued to run fine with existing clients. But any new clients that tried to connect to it for RPC would not work correctly. So when I started a new client, that client made no progress. Despite this, the RS continued to happily heartbeat to the master, so the master did not remove it from the cluster. Note that in this case, we were logging to NFS, and the logs continued to write, but no exceptions shown.