Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.1.0
-
None
Description
Kubernetes supports dynamic allocation via the spark.dynamicAllocation.shuffleTracking.enabled
config, we can add support for stage level scheduling when that is turned on.
Attachments
Issue Links
- relates to
-
SPARK-33462 ResourceProfile use Int for memory in ExecutorResourcesOrDefaults
- Open
- supercedes
-
SPARK-32661 Spark executors on K8S do not request extra memory for off-heap allocations
- Closed
- links to