[KAFKA-4430] Broker logging "Topic and partition to exceptions: [topic,6] -> kafka.common.MessageSizeTooLargeException"

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 0.9.0.1
• Component/s: core
• Labels: newbie
• Environment: Production
• Flags: Important
• Assignee: Unassigned
• Reporter: Srinivas Dhruvakumar
• Created: 22/Nov/16 02:19
• Updated: 10/Feb/17 20:06
• Resolved: 10/Feb/17 20:06

Description

I have a setup as below
DC Kafka
Mirrormaker
Aggregate Kafka
Here is the following settings. I have set the max.message.bytes to 1M Bytes on DC and AGG kafka side. Mirrormaker producer settings – batch.size is set to 500 K Bytes and max.request.size is set to 1 M Byte and ack to 0 , compression-> gzip .
However on the Aggregate Kafka I get the following exception

Closing connection due to error during produce request with correlation id 414156659 from client id producer-1 with ack=0
Topic and partition to exceptions: [topic1,6] -> kafka.common.MessageSizeTooLargeException

Is this a bug or why would this happen. I have configured mirrormaker to send messages less than 1 M Bytes . Are the messages getting dropped ? Under what circumstances this error occurs ?