[YARN-8478] The capacity scheduler logs too frequently seriously affecting performance

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Component/s: capacity scheduler, capacityscheduler
• Assignee: YunFan Zhou
• Reporter: YunFan Zhou
• Created: 29/Jun/18 06:05
• Updated: 30/Jun/18 04:52
• Resolved: 30/Jun/18 04:52

Description

The capacity scheduler logs too frequently, seriously affecting performance.

As a result of our test that the scheduling speed of capacity scheduler is difficult to reach 5000/s in the production scenario.

And it will soon reach the log bottleneck.

My current work is to change many log levels from INFO to DEBUG level.

wangda leftnoteasy Any suggestion?

Issue Links

• Blocked: YARN-5716 Add global scheduler interface definition and update CapacityScheduler to use it.
• is duplicated by: YARN-8479 The capacity scheduler logs too frequently seriously affecting performance