Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-35443

Mark K8s secrets and config maps as immutable

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: Kubernetes
    • Labels:
      None

      Description

      Kubernetes supports marking secrets and config maps as immutable to gain performance. 

      https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable

      https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable

      For K8s clusters that run many thousands of Spark applications, this can yield significant reduction in load on the kube-apiserver.

      From the K8s docs:

      For clusters that extensively use Secrets (at least tens of thousands of unique Secret to Pod mounts), preventing changes to their data has the following advantages:

      • protects you from accidental (or unwanted) updates that could cause applications outages
      • improves performance of your cluster by significantly reducing load on kube-apiserver, by closing watches for secrets marked as immutable.

       

      For any secrets and config maps we create in Spark that are immutable, we could mark them as immutable by including the following when building the secret/config map

      .withImmutable(true)
      

      This feature has been supported in K8s as beta since K8s 1.19 and as GA since K8s 1.21

        Attachments

          Activity

            People

            • Assignee:
              ashrayjain Ashray Jain
              Reporter:
              ashrayjain Ashray Jain
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: