[SPARK-45175] download krb5.conf from remote storage in spark-submit on k8s

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.4.1
• Component/s: Kubernetes
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Qian Sun
• Created: 15/Sep/23 03:17
• Updated: 22/Sep/23 16:57

Description

krb5.conf currently only supports the local file format. Tenants would like to save this file on their own servers and download it during the spark-submit phase for better implementation of multi-tenant scenarios. The proposed solution is to use the downloadFile  function[1], similar to the configuration of spark.kubernetes.driver/executor.podTemplateFile

 

[1]https://github.com/apache/spark/blob/822f58f0d26b7d760469151a65eaf9ee863a07a1/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/PodTemplateConfigMapStep.scala#L82C24-L82C24

Issue Links

• links to: GitHub Pull Request #42943