[KAFKA-4199] When a window store segment is dropped we should also clear any corresponding cached entries

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.10.1.0
• Component/s: streams
• Labels: architecture
• Assignee: Unassigned
• Reporter: Damian Guy
• Created: 20/Sep/16 14:21
• Updated: 11/Nov/16 20:44

Description

In KIP-63 we introduced a CachingWindowStore, but it currently doesn't have a way to be informed when the underlying store drops a segment. In an ideal world, when a segment is dropped we'd also remove the corresponding entries from the cache.

Firstly, we need to understand if it is an issue if they don't get dropped. They will naturally be evicted when the cache becomes full, but this could impact other stores in the thread. i.e., what if any performance impact exists?

If we find there is an unacceptable performance penalty we might need to add a callback to the WindowStore API such that we can be notified when segments are removed