[YARN-7672] hadoop-sls can not simulate huge scale of YARN

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: zhangshilong
• Reporter: zhangshilong
• Created: 19/Dec/17 06:53
• Updated: 18/Apr/20 09:06

Description

Our YARN cluster scale to nearly 10 thousands nodes. We need to do scheduler pressure test.
Using SLS,we start 2000+ threads to simulate NM and AM. But cpu.load very high to 100+. I thought that will affect performance evaluation of scheduler.
So I thought to separate the scheduler from the simulator.
I start a real RM. Then SLS will register nodes to RM,And submit apps to RM using RM RPC.

Issue Links

• is cloned by: YARN-10238 CLONE - hadoop-sls can not simulate huge scale of YARN