[KAFKA-13356] Use "delete" retention policy only for stream-stream join windowed stores

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: streams
• Assignee: Matthias J. Sax
• Reporter: Guozhang Wang
• Created: 06/Oct/21 18:11
• Updated: 07/Oct/21 02:48
• Resolved: 07/Oct/21 02:48

Description

Today stream-stream join associated window stores, like any other window stores, use "delete,compact" as their retention policies. However, since today we add sequence number to disable de-duplication of keys, "compaction" would never be able to compact any keys, but only result in 1) CPU waste on the cleaner thread on brokers, 2) some additional feature of brokers that relies on "delete" policy to not be able to apply.

Until we change the store format potentially in the future to not use sequence number for disable de-duping, we could consider just changing the policy to "delete" for stream-stream join's window store for now.

Issue Links

• duplicates: KAFKA-13345 WindowStore should use "delete" policy when duplicates are enabled
• links to: GitHub PR#11380