[KAFKA-7803] Kafka Streams internal topics config is not updated when the code is changed

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: streams
• Assignee: Unassigned
• Reporter: Tim Van Laer
• Created: 09/Jan/19 09:59
• Updated: 08/Apr/21 20:18

Description

Considering the following state store definition:

ImmutableMap<String, String> changelogTopicConfig = new ImmutableMap.Builder<String, String>()
    .put(TopicConfig.SEGMENT_BYTES_CONFIG, String.valueOf(100 * 1024 * 1024))
    .build();

builder.addStateStore(
    Stores.keyValueStoreBuilder(Stores.persistentKeyValueStore("store-example"), Serdes.String(), Serdes.String())
        .withLoggingEnabled(changelogTopicConfig)
);

The configuration for a changelog topic (segment size, max message size...) is used when Kafka Streams create the internal topic (See InternalTopicManager). If I later decide to increase the segment size, I would update the value in the code. However Kafka Streams currently won't apply this code change to the internal topic config. This causes a confusing state where the code is different from the actual runtime.  

It would be convenient if Kafka Streams could reflect those changes to the internal topic by updating the topic configuration. 

Issue Links

• relates to: KAFKA-7591 Changelog retention period doesn't synchronise with window-store size