Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4631

Refresh consumer metadata more frequently for unknown subscribed topics

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.2.1, 0.11.0.0
    • Component/s: consumer
    • Labels:
      None

      Description

      By default, the consumer refreshes metadata every 5 minutes. In testing, it can often happen that a topic is created at about the same time that the consumer is started. In the worst case, creation finishes after the consumer fetches metadata, and the test must wait 5 minutes for the consumer to refresh metadata in order to discover the topic. To address this problem, users can decrease the metadata refresh interval, but this means more frequent refreshes even after all topics are known. An improvement would be to internally let the consumer fetch metadata more frequently when the consumer encounters unknown topics. Perhaps every 5-10 seconds would be reasonable, for example.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rsivaram Rajini Sivaram
                Reporter:
                hachikuji Jason Gustafson
              • Votes:
                2 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: