[KAFKA-4546] a consumer could miss tombstone when leader changes during the reads

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.10.0.0, 0.10.1.0, 0.11.0.0, 1.0.0
• Component/s: consumer
• Assignee: Unassigned
• Reporter: Jun Rao
• Created: 15/Dec/16 02:33
• Updated: 19/Jul/18 20:19

Description

Currently, in a compacted topic, we delay the removal of a tombstone by delete.retention.ms to avoid the case that a consumer reads a non-tombstone before the tombstone, then tombstone is removed and the consumer misses the tombstone.

However, tombstones are removed independently in different replicas. Supposed that a tombstone is already removed in the follower, but not in the leader. A consumer could read a non-tombstone message on the leader, then the leader changes, and the consumer will miss the tombstone on the new leader since it's already deleted.