Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3007

Implement max.poll.records for new consumer (KIP-41)

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: 0.10.0.0
    • Component/s: consumer
    • Labels:
      None

      Description

      Currently, the consumer.poll(timeout)

      returns all messages that have not been acked since the last fetch
      The only way to process a single message, is to throw away all but the first message in the list
      This would mean we are required to fetch all messages into memory, and this coupled with the client being not thread-safe, (i.e. we cannot use a different thread to ack messages, makes it hard to consume messages when the order of message arrival is important, and a large number of messages are pending to be consumed)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                aartigupta aarti gupta
              • Votes:
                1 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: