[KAFKA-10794] Replica leader election is too slow in the case of too many partitions

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0, 2.5.1
• Fix Version/s: 2.8.0
• Component/s: core
• Assignee: limeng
• Reporter: limeng
• Created: 02/Dec/20 03:12
• Updated: 18/Dec/20 08:25
• Resolved: 03/Dec/20 02:38

Description

There is more than 6000 topics and 300 brokers in my kafka cluster, and we frequently run kafka-preferred-replica-election.sh to rebalance our cluster. But the reblance process spendes too more time and cpu resource like the picture blow.

We find that the function:'controllerContext.allPartitions' is invoked too many times.

 

Issue Links

• links to: GitHub Pull Request #9675