Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.1.0
-
None
-
None
Description
This is an umbrella ticket for work on adding support for dynamic resource allocation into the Kubernetes mode. This requires a Kubernetes-specific external shuffle service. The feature is available in our fork at github.com/apache-spark-on-k8s/spark.
Attachments
Issue Links
- is blocked by
-
SPARK-25299 Use remote storage for persisting shuffle data
- Open
- is duplicated by
-
SPARK-25796 Enable external shuffle service for kubernetes mode.
- Resolved
- is related to
-
SPARK-27963 Allow dynamic allocation without an external shuffle service
- Resolved
-
SPARK-28487 K8S pod allocator behaves poorly with dynamic allocation
- Resolved
- links to