Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2204

Scheduler for Mesos in fine-grained mode launches tasks on wrong executors

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.0.1, 1.1.0
    • Component/s: Mesos
    • Labels:
      None

      Description

      MesosSchedulerBackend.resourceOffers(SchedulerDriver, List[Offer]) is assuming that TaskSchedulerImpl.resourceOffers(Seq[WorkerOffer]) is returning task lists in the same order as the offers it was passed, but in the current implementation TaskSchedulerImpl.resourceOffers shuffles the offers to avoid assigning the tasks always to the same executors. The result is that the tasks are launched on the wrong executors.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                srainville Sebastien Rainville
                Reporter:
                srainville Sebastien Rainville
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: