Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21834

Incorrect executor request in case of dynamic allocation

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.1.2, 2.2.1, 2.3.0
    • Component/s: Scheduler
    • Labels:
      None

      Description

      killExecutor api currently does not allow killing an executor without updating the total number of executors needed. In case of dynamic allocation is turned on and the allocator tries to kill an executor, the scheduler reduces the total number of executors needed ( see https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala#L635) which is incorrect because the allocator already takes care of setting the required number of executors itself.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sitalkedia@gmail.com Sital Kedia
                Reporter:
                sitalkedia@gmail.com Sital Kedia
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: