[SPARK-25825] Kerberos Support for Long Running Jobs in Kubernetes

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Won't Do
• Affects Version/s: 3.0.0
• Component/s: Kubernetes, Spark Core
• Assignee: Unassigned
• Reporter: Ilan Filonenko
• Created: 24/Oct/18 19:18
• Updated: 17/May/20 18:24
• Resolved: 25/Jan/19 23:43

Description

When provided with a --keytab and --principal combination, there is an expectation that Kubernetes would leverage the Driver to spin up a renewal thread to handle token renewal. However, in the case that a --keytab and --principal are not provided and instead a secretName and secretItemKey is provided, there should be an option to specify a config that specifies that a external renewal service exists. The driver should, therefore, be responsible for discovering changes to the secret and send the updated token data to the executor with the UpdateDelegationTokens message. Thereby enabling token renewal given just a secret in addition to the traditional use-case via --keytab and --principal

Issue Links

• is a child of: SPARK-25826 Kerberos Support in Kubernetes resource manager
• links to: [Github] Pull Request #22915 (ifilonenko)
• links to: [Github] Pull Request #22915 (ifilonenko)
• links to: GitHub Pull Request #22915