[YARN-10598] CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to extend the creation type with additional information

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: capacity scheduler
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Benjamin Teke
• Reporter: Benjamin Teke
• Created: 26/Jan/21 14:17
• Updated: 12/Feb/24 01:11
• Resolved: 27/Jan/21 17:15

Description

Under this umbrella (YARN-10496), weight-mode has been implemented for CS with YARN-10504.

Auto-queue creation has been also implemented with YARN-10506.

Connected to this effort, we would like to expose the type of the queue with the RM's /scheduler REST endpoint.

To extend/modify the values added in YARN-10581 these 3 fields will describe a queue:

• queueType : parent/leaf
• creationMethod : static/dynamicLegacy/dynamicFlexible
• autoCreationEligibility : off/legacy/flexible

After this change here are some example cases:

• Static parent queue which has the auto-creation-enabled-v2 false:
  • queueType : parent
  • creationMethod : static
  • autoCreationEligibility : off
• Static managed parent (can have dynamic children):
  • queueType : parent
  • creationMethod : static
  • autoCreationEligibility : legacy
• Legacy auto-created leaf queue (cannot have children):
  • queueType : leaf
  • creationMethod : dynamicLegacy
  • autoCreationEligibility : off
• Auto-created (v2) parent queue, (implicitly) auto-creation-enabled-v2 true: 
  • queueType : parent
  • creationMethod : dynamicFlexible
  • autoCreationEligibility : flexible
• Auto-created (v2) leaf queue (cannot have children):
  • queueType : leaf
  • creationMethod : dynamicFlexible
  • autoCreationEligibility : off

Issue Links

• Dependency: YARN-10579 CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include weight values for queues
• is a clone of: YARN-10581 CS Flexible Auto Queue Creation: Modify RM /scheduler endpoint to include queue creation type for queues