[KAFKA-7129] Dynamic default value for number of thread configuration

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: core
• Assignee: Unassigned
• Reporter: Damien Gasparina
• Created: 03/Jul/18 08:21
• Updated: 03/Jul/18 09:34

Description

There are properties in the broker to change the number of thread of a component (e.g. num.replica.fetchers or num.network.threads). After discussing with astubbs, it seems that the default values are optimized for an 8 CPU machine and might not be optimized for larger machine (e.g. 48 cores). 

For those larger machine, an admin need to tune them to be able to use all resources of the host.

Having dynamic default value (e.g. num.replica.fetchers = ceil(number of core / 8), etc...) instead of static (e.g. num.replica.fetchers =1) could be a more efficient strategy to have default values optimized for different kind of deployment.