[SPARK-26194] Support automatic spark.authenticate secret in Kubernetes backend

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Kubernetes, Spark Core
• Assignee: Marcelo Masiero Vanzin
• Reporter: Marcelo Masiero Vanzin
• Created: 28/Nov/18 00:37
• Updated: 17/May/20 18:24
• Resolved: 06/Dec/18 22:18

Description

Currently k8s inherits the default behavior for spark.authenticate, which is that the user must provide an auth secret.

k8s doesn't have that requirement and could instead generate its own unique per-app secret, and propagate it to executors.

Issue Links

• is related to: SPARK-26301 Consider switching from putting secret in environment variable directly to using secret reference
• is related to: SPARK-26239 Add configurable auth secret source in k8s backend
• links to: [Github] Pull Request #23174 (vanzin)
• links to: [Github] Pull Request #23174 (vanzin)