[YARN-10622] Fix preemption policy to exclude childless ParentQueues

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: capacity scheduler
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Andras Gyori
• Reporter: Andras Gyori
• Created: 10/Feb/21 13:11
• Updated: 12/Feb/24 01:17
• Resolved: 15/Feb/21 13:51

Description

ProportionalCapacityPreemptionPolicy selects the potential LeafQueues to be preempted by this logic:

private Set<String> getLeafQueueNames(TempQueuePerPartition q) {
    // If its a ManagedParentQueue, it might not have any children
    if ((q.children == null || q.children.isEmpty())
        && !(q.parentQueue instanceof ManagedParentQueue)) {
      return ImmutableSet.of(q.queueName);
    }

    Set<String> leafQueueNames = new HashSet<>();
    for (TempQueuePerPartition child : q.children) {
      leafQueueNames.addAll(getLeafQueueNames(child));
    }

    return leafQueueNames;
  }

This, however does not take childless ParentQueues (which was introduced in YARN-10596) into account. 

A childless ParentQueue will throw a NPE in FifoCandidatesSelector#selectCandidates:

        LeafQueue leafQueue = preemptionContext.getQueueByPartition(queueName,
          RMNodeLabelsManager.NO_LABEL).leafQueue;

TempQueuePerPartition has a leafQueue member variable, which is null, if the queue is not a LeafQueue. In case of childless ParentQueue, it is null, but its name is present in the leafQueueNames as stated before.