[YARN-6361] FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues - ASF JIRA

Add vote

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Patch Available
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
None

Description

FSLeafQueue.fetchAppsWithDemand sorts the applications by the current policy. Most of the time is spent in FairShareComparator.compare. We could improve this by doing the calculations outside the sort loop (O(n)) and we sorted by a fixed number inside instead O(n*log(n)). This could be an performance issue when there are huge number of applications in a single queue. The attachments shows the performance impact when there are 10k applications in one queue.