[KAFKA-1196] java.lang.IllegalArgumentException Buffer.limit on FetchResponse.scala + 33

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.10.1.0
• Component/s: consumer
• Labels: newbie
• Environment: running java 1.7, linux and kafka compiled against scala 2.9.2
• Assignee: Ewen Cheslack-Postava
• Reporter: Gerrit Jansen van Vuuren
• Created: 31/Dec/13 16:15
• Updated: 26/Sep/16 03:48
• Resolved: 26/Sep/16 03:48

Description

I have 6 topics each with 8 partitions spread over 4 kafka servers.
the servers are 24 core 72 gig ram.

While consuming from the topics I get an IlegalArgumentException and all consumption stops, the error keeps on throwing.

I've tracked it down to FectchResponse.scala line 33

The error happens when the FetchResponsePartitionData object's readFrom method calls:
messageSetBuffer.limit(messageSetSize)

I put in some debug code the the messageSetSize is 671758648, while the buffer.capacity() gives 155733313, for some reason the buffer is smaller than the required message size.

I don't know the consumer code enough to debug this. It doesn't matter if compression is used or not.

Issue Links

• is related to: KAFKA-2063 Bound fetch response size (KIP-74)