[HBASE-5939] Add an autorestart option in the start scripts

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.95.2
• Fix Version/s: 0.95.0
• Component/s: master, regionserver, scripts
• Hadoop Flags: Reviewed

Release Note

When launched with autorestart, HBase processes will automatically restart if they are not properly terminated, either by a "stop" command or by a cluster stop. To ensure that it does not overload the system when the server itself is corrupted and the process cannot be restarted, the server sleeps for 5 minutes before restarting if it was already started 5 minutes ago previously. To use it, launch the process with "bin/start-hbase autorestart". This option is not fully compatible with the existing "restart" command: if you ask for a restart on a server launched with autorestart, the server will restart but the next server instance won't be automatically restarted.

• Tags: 0.96notable
• Assignee: Nicolas Liochon
• Reporter: Nicolas Liochon
• Created: 04/May/12 16:27
• Updated: 23/Mar/16 23:12
• Resolved: 11/May/12 23:50

Description

When a binary dies on a server, we don't try to restart it while it would be possible in most cases.

We can have something as:
loop
start
wait
if cleanStop then exit
if already stopped less than 5 minutes ago sleep 5 minute
endloop

This is simple for master & backup master, a little bit more complex for the region server as it can be stopped by a script or by the shutdown procedure.

On a long long term it could allow a restart with exactly the same assignments.

Issue Links

• is related to: HBASE-15523 enhance hbase-daemon.sh to enable autorestart.
• is required by: HBASE-5843 Improve HBase MTTR - Mean Time To Recover