[REEF-1492] On IMRU recovery: if ResultHandler.Dispose() throws exception, IMRU Driver hangs.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.16
• Component/s: REEF
• Labels: FT
• Assignee: Julia Wang
• Reporter: Andrey
• Created: 13/Jul/16 19:22
• Updated: 20/Dec/16 23:08
• Resolved: 20/Dec/16 23:08

Description

IMRU scenario:

• one of the map tasks fails
• Driver triggers shutdown on all tasks
• UpdateTaskHost on shutdown is calling ResultHandler.Dispose()
• resulthandler (in my case WriteResultHandler) throws exeption because there are no results (Update function was never executed)

There are couple questions here:

• WriteResulthandler should handle [no results] situation more gracefully, especially on Dispose()
  Probably logic of copy file should be moved from Dispose() to HandleResult() function.

• UpdateTaskHost should handle exceptions from Dispose() call....result handler can be provided by client, so code can throw.
  In case of Dispose() failure the UpdateTaskHost should probably trigger non-recoverable failure, which in turn triggers Driver failure (right now driver hangs)

Issue Links

• Is contained by: REEF-1223 IMRU Fault Tolerance - restart failed evaluators