[YUNIKORN-1961] User Info annotation incorrect for executor pods

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: shim - kubernetes
• Assignee: Unassigned
• Reporter: Mit Desai
• Created: 07/Sep/23 02:56
• Updated: Yesterday 10:34

Description

UserInfo annotation is getting populated correctly for driver pods. But for the executor pods, spark service account information is populated. Ideally, this should be same as the driver pod where we have the information of the user who originally submitted the job.

Here is the example of pod describe when running a worcount job with 2 executors:

Name:             md-wc-94ec368a6d3b30cd-driver
Namespace:        mdesai
Priority:         0
Service Account:  spark
Node:             *** node1 ***
Start Time:       Wed, 06 Sep 2023 18:21:59 -0700
Labels:           spark-app-selector=spark-c9315eb5d3bf4552b30c77090a328249
                  spark-role=driver
Annotations:      otherAnnotations: omited
                  yunikorn.apache.org/allow-preemption: true
                  yunikorn.apache.org/user.info:
                    {"user":"[e-mail]","groups":["group1","group2","...
Status:           Running 

 

Name:             javawordcount-ca2f0f8a6d3b64e6-exec-1
Namespace:        mdesai
Priority:         0
Service Account:  spark
Node:             *** node2 ***
Start Time:       Wed, 06 Sep 2023 18:22:06 -0700
Labels:           spark-app-selector=spark-c9315eb5d3bf4552b30c77090a328249
                  spark-exec-id=1
                  spark-exec-resourceprofile-id=0
                  spark-role=executor
Annotations:      otherAnnotations: omited
                  yunikorn.apache.org/allow-preemption: true
                  yunikorn.apache.org/user.info:
                    {"user":"system:serviceaccount:mdesai:spark","groups":["system:serviceaccounts","system:serviceaccounts:mdesai","system:authenticated"]} 

 

Name:             javawordcount-ca2f0f8a6d3b64e6-exec-2
Namespace:        mdesai
Priority:         0
Service Account:  spark
Node:             *** node3 ***
Start Time:       Wed, 06 Sep 2023 18:22:07 -0700
Labels:           spark-app-selector=spark-c9315eb5d3bf4552b30c77090a328249
                  spark-exec-id=2
                  spark-exec-resourceprofile-id=0
                  spark-role=executor
Annotations:      otherAnnotations: omited
                  yunikorn.apache.org/allow-preemption: true
                  yunikorn.apache.org/user.info:
                    {"user":"system:serviceaccount:mdesai:spark","groups":["system:serviceaccounts","system:serviceaccounts:mdesai","system:authenticated"]}

Issue Links

• relates to: YUNIKORN-1851 Don't allow similar app ids by different users