[SPARK-26295] [K8S] serviceAccountName is not set in client mode

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.4.0
• Component/s: Kubernetes, Spark Core
• Labels: bulk-closed
• Assignee: Unassigned
• Reporter: Adrian Tanase
• Created: 06/Dec/18 12:29
• Updated: 25/May/21 01:51
• Resolved: 25/May/21 01:42

Description

When deploying spark apps in client mode (in my case from inside the driver pod), one can't specify the service account in accordance to the docs (https://spark.apache.org/docs/latest/running-on-kubernetes.html#rbac).

The property spark.kubernetes.authenticate.driver.serviceAccountName is most likely added in cluster mode only, which would be consistent with spark.kubernetes.authenticate.driver being the cluster mode prefix.

We should either inject the service account specified by this property in the client mode pods, or specify an equivalent config: spark.kubernetes.authenticate.serviceAccountName

 This is the exception:

Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "..." is forbidden: User "system:serviceaccount:mynamespace:default" cannot get pods in the namespace "mynamespace"

The expectation was to see the user mynamespace:spark based on my submit command.

My current workaround is to create a clusterrolebinding with edit rights for the mynamespace:default account.

Issue Links

• duplicates: SPARK-28360 The serviceAccountName configuration item does not take effect in client mode.