[SPARK-2204] Scheduler for Mesos in fine-grained mode launches tasks on wrong executors

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.0.1, 1.1.0
• Component/s: Mesos
• Target Version/s: 1.0.1, 1.1.0
• Assignee: Sebastien Rainville
• Reporter: Sebastien Rainville
• Created: 19/Jun/14 20:53
• Updated: 13/Aug/14 09:42
• Resolved: 25/Jun/14 20:21

Description

MesosSchedulerBackend.resourceOffers(SchedulerDriver, List[Offer]) is assuming that TaskSchedulerImpl.resourceOffers(Seq[WorkerOffer]) is returning task lists in the same order as the offers it was passed, but in the current implementation TaskSchedulerImpl.resourceOffers shuffles the offers to avoid assigning the tasks always to the same executors. The result is that the tasks are launched on the wrong executors.

Issue Links

• relates to: SPARK-2269 Clean up and add unit tests for resourceOffers in MesosSchedulerBackend