[KAFKA-4887] Enabling caching on a persistent window store breaks support for duplicate insertion

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.2.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: Elias Levy
• Created: 11/Mar/17 23:06
• Updated: 15/May/20 02:02
• Resolved: 15/May/20 02:02

Description

CachingWindowStore and RocksDBWindowStore interact badly when duplicate insertion support is enabled by passing true as the fourth argument to windowed in the state store supplier.

When the feature is enabled, RocksDBWindowStore correct handles duplicates by assigning a unique sequence number to each element on insertion and using the number within the key.

When caching is enabled by calling enableCaching on the supplier, CachingWindowStore fails to the the same. Thus, of multiple values inserted with the same key, only the last one survives.

Issue Links

• is fixed by: KAFKA-9921 Caching is not working properly with WindowStateStore when retaining duplicates