[SPARK-35443] Mark K8s secrets and config maps as immutable

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: Kubernetes
• Assignee: Ashray Jain
• Reporter: Ashray Jain
• Created: 19/May/21 11:22
• Updated: 20/May/21 04:27
• Resolved: 20/May/21 04:26

Description

Kubernetes supports marking secrets and config maps as immutable to gain performance. 

https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable

https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable

For K8s clusters that run many thousands of Spark applications, this can yield significant reduction in load on the kube-apiserver.

From the K8s docs:

For clusters that extensively use Secrets (at least tens of thousands of unique Secret to Pod mounts), preventing changes to their data has the following advantages:

• protects you from accidental (or unwanted) updates that could cause applications outages
• improves performance of your cluster by significantly reducing load on kube-apiserver, by closing watches for secrets marked as immutable.

 

For any secrets and config maps we create in Spark that are immutable, we could mark them as immutable by including the following when building the secret/config map

.withImmutable(true)

This feature has been supported in K8s as beta since K8s 1.19 and as GA since K8s 1.21

Issue Links

• links to: [Github] Pull Request #32588 (ashrayjain)