[KAFKA-10897] kafka quota optimization

• Type: Improvement
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 2.7.0
• Component/s: admin, clients, config, consumer, core
• Assignee: Kahn Cheny
• Reporter: yangyijun
• Reviewer: Angel Alvarez Pascua
• Created: 01/Jan/21 11:10
• Updated: 21/Dec/23 16:55

Description

1.The current quota dimensions is as follows:

/config/users/<user>/clients/<client-id>
/config/users/<user>/clients/<default>
/config/users/<user>
/config/users/<default>/clients/<client-id>
/config/users/<default>/clients/<default>
/config/users/<default>
/config/clients/<client-id>
/config/clients/<default>

2. Existing problems:

 

2.1.The quota dimensions is not fine enough.
2.2.When multiple users on the same broker produce and consume a large amount of data at the same time, if you want the broker to run normally, you must make the sum of all user quota byte not exceed the upper throughput limit of the broker.
2.3.Even if all the user rate does not reach the upper limit of the broker, but all the user rate is concentrated on a few disks and exceeds the read-write load of the disk, all the produce and consume requests will be blocked.
2.4.Sometimes it's just one topic rate increase sharply under the user, so we just need to limit the increase sharply topics.

 

3. Suggestions for improvement

3.1. Add the upper limit of single broker quota byte.
3.2. Add the upper limit of single disk quota byte on the broker.
3.3. Add topic quota dimensions.