Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4133

Provide a configuration to control consumer max in-flight fetches

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: consumer
    • Labels:

      Description

      With KIP-74, we now have a good way to limit the size of fetch responses, but it may still be difficult for users to control overall memory since the consumer will send fetches in parallel to all the brokers which own partitions that it is subscribed to. To give users finer control, it might make sense to add a `max.in.flight.fetches` setting to limit the total number of concurrent fetches at any time. This would require a KIP since it's a new configuration.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mimaison Mickael Maison
                Reporter:
                hachikuji Jason Gustafson
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated: