[REEF-1692] Revert the ignorance for extra Evaluators requested

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.16
• Labels: FT
• Assignee: Julia Wang
• Reporter: Julia Wang
• Created: 05/Jan/17 01:39
• Updated: 20/Jan/17 20:13
• Resolved: 20/Jan/17 20:13

Description

The reason we did REEF-1688 for ignoring extra AllocatedEvaluators was because we thought Yarn might give us additional Evalautors. In fact, it is a bug in REEF code. REEF-1691 has been created to fix it.

The side effect for ignoring extra IAllocatedEvaluator are:

• It hides potential bugs
• Even we choose to ignore the extra IAllocatedEvaluator in EvaluatorManager, this event can be received in other system state rather than WaitingForEvalautor. This is because as soon as we get all the needed Evaluators, we change the system state and start to submit tasks, and the system state can be SubmittingTasks, or ShuttingDown after it. By the time we receive the extra IAllocatedEvaluator event, it is in unexpected state, IMRUDriver will throw exception in those cases.

I would think we should revert the change in REEF-1688 to ensure the integrity of the system.

Issue Links

• is related to: REEF-1223 IMRU Fault Tolerance - restart failed evaluators
• relates to: REEF-1691 Should not request extra evaluators if evaluator failed at WatingForEvaluator state
• relates to: REEF-1688 IMRU driver fails if RM allocates extra evaluators