[KAFKA-4876] WindowStore.fetch(key, 0, Long.MaxValue) is very slow

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.10.1.0, 0.10.2.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: Xavier Léauté
• Created: 09/Mar/17 17:42
• Updated: 07/Apr/17 21:44
• Resolved: 09/Mar/17 17:50

Description

Relates to KAFKA-4851

Querying a window store over the entire time range (i.e. [0, Long.MAX_VALUE] is extremely slow.

Here's a simple benchmark that shows it takes about 400ms to query just a single key with a timestamp as of today's date.

https://github.com/xvrl/kafka/commit/045b94d4c70d730d10ed492efeb0fd85a70757ae#diff-46a87bcb14bccc04c8ad716a043fc78eR150

Benchmark                                    Mode  Cnt    Score    Error   Units
RocksDBWindowStoreTest.benchmarkStoreFetch   avgt   20  401.635 ± 46.130   ms/op

Using timestamps far out in the future (e.g. 0x7a00000000000000L) the fetch operation essentially never completes.

Issue Links

• is duplicated by: KAFKA-4851 SessionStore.fetch(key) is a performance bottleneck