Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-4525

MesosSchedulerBackend.resourceOffers cannot decline unused offers from acceptedOffers

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.2.0, 1.3.0
    • Fix Version/s: 1.2.0
    • Component/s: Mesos
    • Labels:
      None
    • Target Version/s:

      Description

      After resourceOffers function is refactored - SPARK-2269 -, that function doesn't decline unused offers from accepted offers. That's because when driver.launchTasks is called, if that's tasks is empty, driver.launchTask calls the declineOffer(offer.id).

      Invoking this function with an empty collection of tasks declines offers in their entirety (see SchedulerDriver.declineOffer(OfferID, Filters)).

      In branch-1.1, resourcesOffers calls a launchTask function for all offered offers, so driver declines unused resources, however, in current master, at first offers are divided accepted and declined offers by their resources, and delinedOffers are declined explicitly, and offers with task from acceptedOffers are launched by driver.launchTasks, but, offers without from acceptedOfers are not launched with empty task or declined explicitly. Thus, mesos master judges thats offers used by TaskScheduler and there are no resources remaing.

        Attachments

          Activity

            People

            • Assignee:
              jongyoul Jongyoul Lee
              Reporter:
              jongyoul Jongyoul Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: