[YUNIKORN-831] node sorting should check other resources of nodes instead of comparing node id directly

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Fix Version/s: 0.12.1
• Labels: pull-request-available
• Target Version: 0.12.1
• Assignee: Chia-Ping Tsai
• Reporter: Chia-Ping Tsai
• Created: 31/Aug/21 09:13
• Updated: 21/Jan/22 21:48
• Resolved: 16/Sep/21 04:16

Description

from comment: 

https://issues.apache.org/jira/browse/YUNIKORN-807?focusedCommentId=17407164&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17407164

 

It seems to me a small behavior change is introduced by this patch. This patch scores node according to `resources.LargestUsageShare`. However, the largest usage in our cluster is always storage (and all nodes have same storage capacity). It results in the order is always based on `node id`. Before this patch, the order will compare remaining "usage" (vcore, memory, etc) when largest usage is equal.

There are two possible solutions.

1. the nodeScore of nodeRef should keep all usages ([]float64) and the implementation of `Less` function should reuse `resources.CompareShares`
2. add an new config to enable users to control scope. for example, the following config means we should compare vcore instead of largest resource 

nodesortpolicy:
  type: binpacking
  usage: vcore

 

Issue Links

• is superceded by: YUNIKORN-847 Use resource weighting for node sorting
• relates to: YUNIKORN-807 Improve performance of node sorting
• links to: GitHub Pull Request #318