[KAFKA-4402] Kafka Producer's DefaultPartitioner is actually not round robin as said in the code comments "If no partition or key is present choose a partition in a round-robin fashion"

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.10.2.0
• Assignee: Unassigned
• Reporter: Jun Yao
• Created: 13/Nov/16 21:40
• Updated: 09/Jan/17 04:09
• Resolved: 09/Jan/17 04:08

Description

From this code comments, it is said that Kafka client Producer's DefaultPartitioner will do round robin if "no partition or key is present",
https://github.com/apache/kafka/blob/41e676d29587042994a72baa5000a8861a075c8c/clients/src/main/java/org/apache/kafka/clients/producer/internals/DefaultPartitioner.java#L34

from the code it looks trying to do round robin as well, as it maintained a counter and try to increase it every time and then will decide which partition to go to;

However the issue here is the counter is a global counter that is shared by all the topics, so it is actually not round robin per topic and sometimes caused unbalanced routing among different partitions.

Although we can pass a custom implementation of interface "org.apache.kafka.clients.producer.Partitioner", it might be still good to make the default implementation true round robin as comment.

Issue Links

• links to: GitHub Pull Request #2128