[HBASE-8703] [WINDOWS] Timed-out processes exit with non-zero code causing HealthChecker to report incorrectly

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.98.0, 0.95.2
• Hadoop Flags: Reviewed
• Assignee: Enis Soztutar
• Reporter: Enis Soztutar
• Created: 06/Jun/13 21:41
• Updated: 23/Sep/13 19:22
• Resolved: 10/Jun/13 18:36

Description

Hadoop ShellCommandExecutor on timeout, destroys the process, and later checks the exit code. On windows, if process is destroyed, the exit code is non-zero, so ShellCommandExecutor.execute() throws ExitCodeException rather than generic IOException. However, in HealthChecker, on catching ExitCodeException we do not check the timeout.

It is unfortunate that Hadoop does not throw an TimeoutException, but provides .isTimedOut() api. And it seems that on timeout on linux, the throwing of the IOException is coincidental:

2013-06-06 14:17:47,930 WARN  [main] hbase.HealthChecker(79): Caught exception : java.io.IOException: Stream closed

It may not be worth fixing Hadoop for this, but we have to do the correct check anyway.