[KAFKA-13345] WindowStore should use "delete" policy when duplicates are enabled

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.1.0
• Component/s: streams
• Assignee: Matthias J. Sax
• Reporter: Matthias J. Sax
• Created: 04/Oct/21 23:24
• Updated: 15/Oct/21 05:06
• Resolved: 15/Oct/21 05:06

Description

For stream-stream join, windowed stores are configures to retain duplicates. For this case, each key written into the store is unique.

At the same time, windowed store changelog topics are configured with cleanup policy "delete,compact" by default. However, with retained duplicates, and thus effectively unique keys, log compaction does not make sense.

To avoid keeping the broker log compaction thread busy without doing actual useful work for those topics, we should change the topic config to "delete" if duplicates are retained.

Issue Links

• is duplicated by: KAFKA-13356 Use "delete" retention policy only for stream-stream join windowed stores
• links to: GitHub Pull Request #11380