[YUNIKORN-1253] PVCs won't get past WaitForFirstConsumer with Apache Yunikorn

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Delivered
• Fix Version/s: 1.2.0
• Component/s: shim - kubernetes
• Target Version: 1.2.0
• Assignee: Chen Yu Teng
• Reporter: Alex Kalenyuk
• Created: 04/Jul/22 17:46
• Updated: 07/Feb/23 22:33
• Resolved: 22/Dec/22 17:36

Description

It seems that with Apache Yunikorn, WaitForFirstConsumer volume binding storage classes are not supported (not sure if this is intended or not).

This makes it problematic to use storage that is not globally accessible from all nodes:
https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode

To reproduce a simple failure to use a WaitForFirstConsumer binding-backed PVC:
```
kind: Pod
apiVersion: v1
metadata:
  name: test
  namespace: default
  labels:
    app: sleep
    applicationId: "sleep0001-node-selector-test"
    queue: "root.sandbox"
spec:
  schedulerName: yunikorn
  nodeSelector:
    storage/ssd: 'true'
  containers:
    - name: test
      resources:
        limits:
          cpu: 1
          memory: 1G
        requests:
          cpu: 1
          memory: 1G
      image: busybox
      command:
        - sleep
        - '1000000'
      volumeMounts:
        - name: scratch-volume
          mountPath: /data
  volumes:
    - name: scratch-volume
      ephemeral:
        volumeClaimTemplate:
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 1Gi
            storageClassName: hostpath-provisioner
            volumeMode: Filesystem
```
Storage used:
https://github.com/kubevirt/hostpath-provisioner-operator

A similar issue was spotted in:
https://github.com/kubernetes/kubernetes/issues/86262
And this PR seems to introduce the VolumeBinding filter but comments it out:
https://github.com/apache/yunikorn-k8shim/pull/313

 

I might be off with the "Bug" type here so feel free to correct me;
My thinking was that if introducing support for WFFC is trivial, this may make sense to exist in older versions too.

Issue Links

• is fixed by: YUNIKORN-1196 Upgrade K8s shim build dependency to 1.23.12
• is related to: YUNIKORN-1039 add e2e test that uses volumes