Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3007

Implement max.poll.records for new consumer (KIP-41)

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.0.0
    • 0.10.0.0
    • consumer
    • None

    Description

      Currently, the consumer.poll(timeout)

      returns all messages that have not been acked since the last fetch
      The only way to process a single message, is to throw away all but the first message in the list
      This would mean we are required to fetch all messages into memory, and this coupled with the client being not thread-safe, (i.e. we cannot use a different thread to ack messages, makes it hard to consume messages when the order of message arrival is important, and a large number of messages are pending to be consumed)

      Attachments

        Issue Links

          Activity

            People

              hachikuji Jason Gustafson
              aartigupta aarti gupta
              Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: