Uploaded image for project: 'Apache YuniKorn'
  1. Apache YuniKorn
  2. YUNIKORN-1253

PVCs won't get past WaitForFirstConsumer with Apache Yunikorn

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • shim - kubernetes
    • None

    Description

      It seems that with Apache Yunikorn, WaitForFirstConsumer volume binding storage classes are not supported (not sure if this is intended or not).

      This makes it problematic to use storage that is not globally accessible from all nodes:
      https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode

      To reproduce a simple failure to use a WaitForFirstConsumer binding-backed PVC:
      ```
      kind: Pod
      apiVersion: v1
      metadata:
        name: test
        namespace: default
        labels:
          app: sleep
          applicationId: "sleep0001-node-selector-test"
          queue: "root.sandbox"
      spec:
        schedulerName: yunikorn
        nodeSelector:
          storage/ssd: 'true'
        containers:
          - name: test
            resources:
              limits:
                cpu: 1
                memory: 1G
              requests:
                cpu: 1
                memory: 1G
            image: busybox
            command:
              - sleep
              - '1000000'
            volumeMounts:
              - name: scratch-volume
                mountPath: /data
        volumes:
          - name: scratch-volume
            ephemeral:
              volumeClaimTemplate:
                spec:
                  accessModes:
                    - ReadWriteOnce
                  resources:
                    requests:
                      storage: 1Gi
                  storageClassName: hostpath-provisioner
                  volumeMode: Filesystem
      ```
      Storage used:
      https://github.com/kubevirt/hostpath-provisioner-operator

      A similar issue was spotted in:
      https://github.com/kubernetes/kubernetes/issues/86262
      And this PR seems to introduce the VolumeBinding filter but comments it out:
      https://github.com/apache/yunikorn-k8shim/pull/313

       

      I might be off with the "Bug" type here so feel free to correct me;
      My thinking was that if introducing support for WFFC is trivial, this may make sense to exist in older versions too.

      Attachments

        1. case01_schedulingWhenPVisNotCreated.log
          5 kB
          Chen Yu Teng
        2. case01_yunikorn_scheduling_decision.log
          23 kB
          Chen Yu Teng
        3. pv.yaml
          0.4 kB
          Chen Yu Teng
        4. storageclass.yaml
          0.2 kB
          Chen Yu Teng

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Yukali Chen Yu Teng
            akalenyu Alex Kalenyuk

            Dates

              Created:
              Updated:

              Slack

                Issue deployment