[SPARK-48669] K8s resource name prefix follows DNS Subdomain Names rule

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.5.1
• Component/s: k8s
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Xi Chen
• Created: 20/Jun/24 07:57
• Updated: 11/Oct/24 00:23

Description

In SPARK-39614, we extended the allowed name length from 63 to 253 for executor pod and config map.

However, when the pod name is exceeded length 253, we don't truncate it. This leads to error when creating the Spark pods.

Error example:

Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://kubernetes.default.svc.cluster.local:443/api/v1/namespaces/foo/pods. Message: Pod "some-super-long-spark-pod-name-exceeded-length-253-driver" is invalid: metadata.name: Invalid value: "some-super-long-spark-pod-name-exceeded-length-253-driver": must be no more than 253 characters. 

Issue Links

• links to: GitHub Pull Request #47039