Description
Currently, the consumer.poll(timeout)
returns all messages that have not been acked since the last fetch
The only way to process a single message, is to throw away all but the first message in the list
This would mean we are required to fetch all messages into memory, and this coupled with the client being not thread-safe, (i.e. we cannot use a different thread to ack messages, makes it hard to consume messages when the order of message arrival is important, and a large number of messages are pending to be consumed)
Attachments
Issue Links
- is related to
-
KAFKA-2986 Consumer group doesn't lend itself well for slow consumers with varying message size
-
- Resolved
-
- mentioned in
-
Page Loading...