Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-17637

Packed scheduling for Spark tasks across executors

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Incomplete
    • None
    • None
    • Scheduler, Spark Core

    Description

      Currently Spark scheduler implements round robin scheduling for tasks to executors. Which is great as it distributes the load evenly across the cluster, but this leads to significant resource waste in some cases, especially when dynamic allocation is enabled.

      Attachments

        Activity

          People

            zhazhan Zhan Zhang
            zhzhan Zhan Zhang
            Votes:
            1 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: