[KAFKA-4633] Always use regex pattern subscription to avoid auto create topics

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.10.2.0
• Component/s: streams
• Labels: architecture
• Assignee: Guozhang Wang
• Reporter: Guozhang Wang
• Created: 14/Jan/17 00:53
• Updated: 08/Jun/17 07:37
• Resolved: 23/Jan/17 18:54

Description

In KafkaConsumer, a metadata update is requested whenever subscribe(List<String> topics ..) is called. And when such a metadata request is sent to the broker upon the first poll call, it will cause the broker to auto-create any topics that do not exist if the broker-side config topic.auto.create is turned on.

In order to work around this issue until the config is default to false and gradually be deprecated, we will let Streams to always use the other subscribe function with regex pattern, which will send the metadata request with empty topic list and hence won't trigger broker-side auto topic creation.

The side-effect is that the metadata response will be larger, since it contains all the topic infos; but since we only refresh it infrequently this will add negligible overhead.

Issue Links

• links to: GitHub Pull Request #2379