This is a follow up to
SPARK-26194, which aims to add auto-generated secrets similar to the YARN backend.
There's a desire to support different ways to generate and propagate these auth secrets (e.g. using things like Vault). Need to investigate:
- exposing configuration to support that
- changing SecurityManager so that it can delegate some of the secret-handling logic to custom implementations
- figuring out whether this can also be used in client-mode, where the driver is not created by the k8s backend in Spark.