[KAFKA-10493] KTable out-of-order updates are not being ignored

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Won't Fix
• Affects Version/s: 2.6.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: Pedro Gontijo
• Created: 17/Sep/20 06:22
• Updated: 28/Dec/22 23:12
• Resolved: 28/Dec/22 23:11

Description

On a materialized KTable, out-of-order records for a given key (records which timestamp are older than the current value in store) are not being ignored but used to update the local store value and also being forwarded.

I believe the bug is here: https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/state/internals/ValueAndTimestampSerializer.java#L77 It should return true, not false (see javadoc)

The bug impacts here: https://github.com/apache/kafka/blob/2.6.0/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KTableSource.java#L142-L148

I have attached a simple stream app that shows the issue happening.

Thank you!

Issue Links

• depends upon: KAFKA-7061 Enhanced log compaction
• is superceded by: KAFKA-14491 Introduce Versioned Key-Value Stores to Kafka Streams
• links to: GitHub Pull Request #10533
• links to: GitHub Pull Request #12356