Minor Description
The documentation around the producer configurations "batch.size" and "max.request.size", and how they relate to one another, can be confusing.
In reality, the "max.request.size" is a hard limit on each individual record, but the documentation makes it seem this is the maximum size of a request sent to Kafka. If there is a situation where "batch.size" is set greater than "max.request.size" (and each individual record is smaller than "max.request.size") you could end up with larger requests than expected sent to Kafka.
There are a few things that could be considered to make this clearer:
- Improve the documentation to clarify the two producer configurations and how they relate to each other
- Provide a producer check, and possibly a warning, if "batch.size" is found to be greater than "max.request.size"
- The producer could take the minimum of "batch.size" or "max.request.size"
Attachments
Issue Links
- is duplicated by
-
KAFKA-12351 Fix misleading max.request.size behavior
-
- Resolved
-
- links to