[YARN-10813] Set default capacity of root for node labels

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.2.3, 3.3.2
• Fix Version/s: 3.4.0, 3.2.3, 3.3.2
• Component/s: capacity scheduler
• Target Version/s: 3.4.0, 3.2.3, 3.3.2
• Hadoop Flags: Reviewed
• Assignee: Andras Gyori
• Reporter: Andras Gyori
• Created: 09/Jun/21 11:40
• Updated: 11/Feb/24 14:25
• Resolved: 28/Jul/21 12:56

Description

CapacitySchedulerConfiguration#getNonLabeledQueueCapacity handles root in the following way:

if (absoluteResourceConfigured || configuredWeightAsCapacity(
        configuredCapacity)) {
      // Return capacity in percentage as 0 for non-root queues and 100 for
      // root.From AbstractCSQueue, absolute resource will be parsed and
      // updated. Once nodes are added/removed in cluster, capacity in
      // percentage will also be re-calculated.
      return queue.equals("root") ? 100.0f : 0f;
    }

CapacitySchedulerConfiguration#internalGetLabeledQueueCapacity on the other hand does not take root queue into consideration:

if (absoluteResourceConfigured || configuredWeightAsCapacity(
        configuredCapacity)) {
      // Return capacity in percentage as 0 for non-root queues and 100 for
      // root.From AbstractCSQueue, absolute resource, and weight will be parsed
      // and updated separately. Once nodes are added/removed in cluster,
      // capacity is percentage will also be re-calculated.
      return defaultValue;
    }

    float capacity = getFloat(capacityPropertyName, defaultValue);

Due to this, labeled root capacity is 0, which is not set in in AbstractCSQueue#derivedCapacityFromAbsoluteConfigurations, because root is never in Absolute mode.

Issue Links

• is blocked by: YARN-10780 Optimise retrieval of configured node labels in CS queues
• is related to: YARN-10823 Expose all node labels for root without explicit configurations