[YARN-9925] CapacitySchedulerQueueManager allows unsupported Queue hierarchy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.3.0
• Component/s: capacityscheduler
• Release Note: Currently, CapacityScheduler does not allow two queues with same name irrespective of hierarchy or position. For example, root.a.a1 and root.b.a1 are not allowed. root.a.b.c and root.a.d.b.e are not allowed.
• Assignee: Prabhu Joseph
• Reporter: Prabhu Joseph
• Created: 21/Oct/19 19:06
• Updated: 13/Jan/20 17:23
• Resolved: 13/Jan/20 17:23

Description

CapacitySchedulerQueueManager allows unsupported Queue hierarchy. When creating a queue with same name as an existing parent queue name - it has to fail with below.

Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed.Caused by: java.io.IOException: A is moved from:root.A to:root.B.A after refresh, which is not allowed. at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.validateQueueHierarchy(CapacitySchedulerQueueManager.java:335) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.reinitializeQueues(CapacitySchedulerQueueManager.java:180) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitializeQueues(CapacityScheduler.java:762) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.reinitialize(CapacityScheduler.java:473) ... 70 more 

In Some cases, the error is not thrown while creating the queue but thrown at submission of job "Failed to submit application_1571677375269_0002 to YARN : Application application_1571677375269_0002 submitted by user : systest to non-leaf queue : B"

Below scenarios are allowed but it should not

It allows root.A.A1.B when root.B.B1 already exists.
   
1. Add root.A
2. Add root.A.A1
3. Add root.B
4. Add root.B.B1
5. Allows Add of root.A.A1.B 

It allows two root queues:
   
1. Add root.A
2. Add root.B
3. Add root.A.A1
4. Allows Add of root.A.A1.root
	 

Below scenario is handled properly:

It does not allow root.B.A when root.A.A1 already exists.
     
1. Add root.A
2. Add root.B
3. Add root.A.A1
4. Does not Allow Add of root.B.A

This error handling has to be consistent in all scenarios.

Issue Links

• duplicates: YARN-9879 Allow multiple leaf queues with the same name in CapacityScheduler
• is related to: YARN-9772 CapacitySchedulerQueueManager has incorrect list of queues