[YUNIKORN-1953] Can't use Priority directly in placeholder pods

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.0
• Fix Version/s: 1.4.0
• Component/s: shim - kubernetes
• Target Version: 1.4.0
• Assignee: PoAn Yang
• Reporter: PoAn Yang
• Created: 02/Sep/23 07:24
• Updated: 20/Nov/23 16:42
• Resolved: 05/Sep/23 11:58

Description

Placeholder pods can't be created in a job with non-zero value PriorityClass. I got following error in the yunikorn-k8shim:

 

2023-09-02T07:11:57.330Z    ERROR    shim.cache.placeholder    cache/placeholder_manager.go:99    failed to create placeholder pod    {"error": "pods \"tg-group-low-app-low-0\" is forbidden: the integer value of priority (-100) must not be provided in pod spec; priority admission controller computed 0 from the given PriorityClass name"}

We set `priority` in placeholder pods [1]. However, if Priority Admission Controller is enabled, it prevents users from setting this field [2].

 

 

[1] https://github.com/apache/yunikorn-k8shim/blob/8b26c373b4b5ba1c1462cb779a3f467ee0ae6c1d/pkg/cache/placeholder.go#L124

[2] https://github.com/kubernetes/kubernetes/blob/4619f7e9d9d64a0ae0b17270496ca3f7c67d24e8/staging/src/k8s.io/api/core/v1/types.go#L3469-L3475

Following is my test YAML:

apiVersion: scheduling.k8s.io/v1
kind: PriorityClass
metadata:
  name: yk-low
preemptionPolicy: Never
value: -100
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    applicationId: app-low
  name: app-low
spec:
  backoffLimit: 6
  completionMode: NonIndexed
  completions: 1
  parallelism: 1
  suspend: false
  template:
    metadata:
      annotations:
        yunikorn.apache.org/task-group-name: group-low
        yunikorn.apache.org/task-groups: '[{"Name":"group-low","MinMember":1,"MinResource":{"cpu":"100m","memory":"100M"}}]'
      labels:
        applicationId: app-low
        job-name: app-low
      name: test-low-priority
    spec:
      containers:
      - command:
        - sleep
        - "300"
        image: alpine:latest
        imagePullPolicy: IfNotPresent
        name: sleepcontainer
        resources:
          requests:
            cpu: 100m
            memory: 100M
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      priorityClassName: yk-low
      restartPolicy: Never
      schedulerName: yunikorn
      securityContext: {}
      terminationGracePeriodSeconds: 1 

Issue Links

• is related to: YUNIKORN-1921 Gang Scheduled Placeholder Pods Don't Inherit PriorityClasses
• is related to: YUNIKORN-1935 Add Gang Scheduling with Priority e2e test case
• links to: GitHub Pull Request #666