Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4133

Provide a configuration to control consumer max in-flight fetches

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • consumer

    Description

      With KIP-74, we now have a good way to limit the size of fetch responses, but it may still be difficult for users to control overall memory since the consumer will send fetches in parallel to all the brokers which own partitions that it is subscribed to. To give users finer control, it might make sense to add a `max.in.flight.fetches` setting to limit the total number of concurrent fetches at any time. This would require a KIP since it's a new configuration.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            mimaison Mickael Maison
            hachikuji Jason Gustafson

            Dates

              Created:
              Updated:

              Slack

                Issue deployment