Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Incomplete
-
None
-
None
Description
Currently Spark scheduler implements round robin scheduling for tasks to executors. Which is great as it distributes the load evenly across the cluster, but this leads to significant resource waste in some cases, especially when dynamic allocation is enabled.