Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4633

Always use regex pattern subscription to avoid auto create topics

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.2.0
    • Component/s: streams
    • Labels:

      Description

      In KafkaConsumer, a metadata update is requested whenever subscribe(List<String> topics ..) is called. And when such a metadata request is sent to the broker upon the first poll call, it will cause the broker to auto-create any topics that do not exist if the broker-side config topic.auto.create is turned on.

      In order to work around this issue until the config is default to false and gradually be deprecated, we will let Streams to always use the other subscribe function with regex pattern, which will send the metadata request with empty topic list and hence won't trigger broker-side auto topic creation.

      The side-effect is that the metadata response will be larger, since it contains all the topic infos; but since we only refresh it infrequently this will add negligible overhead.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                guozhang Guozhang Wang
                Reporter:
                guozhang Guozhang Wang
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: