[YUNIKORN-520] Gang scheduling waits indefinitely for placeholder pod allocation even where is no quota left in the queue

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10
• Fix Version/s: 0.10
• Component/s: shim - kubernetes
• Labels: pull-request-available
• Target Version: 0.10
• Assignee: Manikandan R
• Reporter: Ayub Pathan
• Created: 28/Jan/21 02:03
• Updated: 21/Jan/22 21:48
• Resolved: 24/Feb/21 01:37

Description

• Create a namespace(ns.yaml with quota.
• Submit a job(job.yaml where the placeholder pods resource requests are more than queue quota.

The YK waits indefinitely for the placeholder pods minMember limit to be satisfied. Gang scheduling can action to avoid this indefinite lock situation, incase of resource limits.

 

NAME↑                       READY STATUS   RS CPU MEM %CPU/R %MEM/R %CPU/L %MEM/L IP                NODE                                              QOS  AGE      │
│ batch-sleep-job-4-hzc49     0/0   Pending   0 n/a n/a    n/a    n/a    n/a    n/a n/a               n/a                                               BU   4h28m    │
│ batch-sleep-job-4-jmqbf     0/0   Pending   0 n/a n/a    n/a    n/a    n/a    n/a n/a               n/a                                               BU   4h28m    │
│ batch-sleep-job-4-kg7hl     0/0   Pending   0 n/a n/a    n/a    n/a    n/a    n/a n/a               n/a                                               BU   4h28m    │
│ tg-tg1-batch-sleep-job-4-0  1/1   Running   0 n/a n/a    n/a    n/a    n/a    n/a 100.100.134.233   ip-10-192-136-201.ca-central-1.compute.internal   BU   4h28m    │
│ tg-tg1-batch-sleep-job-4-1  1/1   Running   0 n/a n/a    n/a    n/a    n/a    n/a 100.100.87.75     ip-10-192-159-151.ca-central-1.compute.internal   BU   4h28m    │
│ tg-tg1-batch-sleep-job-4-2  0/0   Pending   0 n/a n/a    n/a    n/a    n/a    n/a n/a               n/a                                               BU   4h28m    │ 

Issue Links

• relates to: YUNIKORN-550 taskgroup cpu and memory resources are not scaled correctly
• links to: GitHub Pull Request #234