[REEF-1224] IMRU Fault Tolerance - Separate Data downloading from Task injection

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.15
• Component/s: IMRU, REEF.NET
• Labels: FT
• Assignee: Dhruv Mahajan
• Reporter: Julia Wang
• Created: 28/Feb/16 00:51
• Updated: 22/Apr/16 17:52
• Resolved: 22/Apr/16 17:52

Description

Currently in IMRU, data downloading happens during the Task injection. It couples the data and Task object. In Fault tolerant case, we would like to only resubmit a task but use the data that have been downloaded, That requires us to decouple those two portions. For example, data downloading portion can be attached to Context, and we can then resubmit a task on the same context.

Issue Links

• Blocked: REEF-1278 IFailedEvaluator message leads to shutting down of JAVA side driver
• blocks: REEF-1251 IMRU Driver handlers for Fault Tolerant
• contains: REEF-1339 Adding IInputPartition.Cache() for data download and cache
• Is contained by: REEF-1223 IMRU Fault Tolerance - restart failed evaluators
• is related to: REEF-1260 Adding a sample for Context Start handler