Details
-
Bug
-
Status: Closed
-
Blocker
-
Resolution: Fixed
-
None
-
None
-
None
Description
To reproduce the issue:
1. Start 1 zookeeper
2. Start 1 broker
3. Send some messages
4. Start SimpleConsumerPerformance to consume messages. The only way to consume all messages is to set the fetch-size to be greater than the log segment file size.
5. This output shows that SimpleConsumerPerformance consumes only 6 messages:
$ bin/kafka-run-class.sh kafka.perf.SimpleConsumerPerformance --server kafka://host1:9092 --topic topic_001 --fetch-size 2048 --partition 0
start.time, end.time, fetch.size, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec
2013-01-21 15:09:21:124, 2013-01-21 15:09:21:165, 2048, 0.0059, 0.1429, 6, 146.3415
6. This output shows that ConsoleConsumer consumes all 5500 messages (same test as the above)
$ bin/kafka-run-class.sh kafka.consumer.ConsoleConsumer --zookeeper host2:2181 --topic topic_001 --consumer-timeout-ms 5000 --formatter kafka.consumer.ChecksumMessageFormatter --from-beginning | grep ^checksum | wc -l
Consumed 5500 messages
5500