[YARN-10891] Extend QueueInfo with max-parallel-apps in CapacityScheduler

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: capacity scheduler
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: Tamas Domok
• Reporter: Tamas Domok
• Created: 19/Aug/21 06:14
• Updated: 27/Aug/21 21:11
• Resolved: 27/Aug/21 21:11

Description

Add max-parallel-apps to the Cluster Scheduler API's response and extend the Yarn-API's QueueInfoProto with the max-parallel-apps property.

 

The REST api can be tested with:

curl "http://localhost:8088/ws/v1/cluster/scheduler" | jq 

 

The protobuf api can be tested with the yarn client:

yarn queue --status root.queue.foo
Queue Information :
Queue Name : foo
Queue Path : root.queue.foo
	State : RUNNING
	Capacity : 75.00%
	Current Capacity : .00%
	Maximum Capacity : 100.00%
	Weight : -1.00
	Maximum Parallel Apps : 9
	Default Node Label expression : <DEFAULT_PARTITION>
	Accessible Node Labels : *
	Preemption : disabled
	Intra-queue Preemption : disabled 

 

About the max-parallel-apps:

Maximum number of applications that can run at the same time. Unlike to maximum-applications, application submissions are not rejected when this limit is reached. Instead they stay in ACCEPTED state until they are eligible to run. This can be set for all queues with yarn.scheduler.capacity.max-parallel-apps and can also be overridden on a per queue basis by setting yarn.scheduler.capacity.<queue-path>.max-parallel-apps. Integer value is expected. By default, there is no limit.

Issue Links

• links to: GitHub Pull Request #3314