[YARN-6242] [Umbrella] Miscellaneous Scheduler Performance Improvements

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Wangda Tan
• Created: 27/Feb/17 21:47
• Updated: 20/Mar/17 18:19

Description

There're some performance issues of scheduler. YARN-3091 is majorly targeted to solve locking issues of scheduler, Let's use this JIRA to track non-locking issues.

Issue Links

• relates to: YARN-5479 FairScheduler: Scheduling performance improvement

Sub-Tasks

• CapacityScheduler: canAssignToThisQueue should be called only when necessary: Resolved
• Add FinalResource object to reduce overhead of Resource class instancing: Open
• Make Collections.sort() more efficient by caching resource usage: Resolved
• FairScheduler: FSLeafQueue.fetchAppsWithDemand CPU usage is high with big queues: Patch Available
• Reduce Resource instance overhead via non-PBImpl: Resolved
• Avoid locking overhead for NO_LABEL lookups: Resolved
• Eliminate double-copy of child queues in canAssignToThisQueue: Resolved
• Improve performance of AssignmentInformation datastructures: Resolved
• ChildQueue can override AbstractCSQueue.hasChildQueues to return false always: Patch Available
• CapacityScheduler: Improvements to assignContainers, avoid unnecessary canAssignToUser/Queue calls: Resolved
• Filter non-existent resource requests actively: Patch Available