[KAFKA-7591] Changelog retention period doesn't synchronise with window-store size

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: streams
• Assignee: Unassigned
• Reporter: Jon Bates
• Created: 05/Nov/18 11:25
• Updated: 08/Apr/21 20:18

Description

When a new windowed state store is created, the associated changelog topic's `retention.ms` value is set to `window-size + CHANGELOG_ADDITIONAL_RETENTION_MS`

Expected Behaviour

If the window-size is updated, the changelog topic's `retention.ms` config should be updated to reflect the new size

Actual Behaviour

The changelog-topic's `retention.ms` setting is not amended, resulting in possible loss of data upon application restart

 

n.b. Although it is easy to update changelog topic config, I logged this as `major` due to the potential for data-loss for any user of Kafka-Streams who may not be intimately aware of the relationship between a windowed store and the changelog config

Issue Links

• is related to: KAFKA-7803 Kafka Streams internal topics config is not updated when the code is changed