Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4529

tombstone may be removed earlier than it should

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.1.0
    • Fix Version/s: 0.10.1.1
    • Component/s: None
    • Labels:
      None

      Description

      As part of KIP-33, we introduced a regression on how tombstone is removed in a compacted topic. We want to delay the removal of a tombstone to avoid the case that a reader first reads a non-tombstone message on a key and then doesn't see the tombstone for the key because it's deleted too quickly. So, a tombstone is supposed to only be removed from a compacted topic after the tombstone is part of the cleaned portion of the log after delete.retention.ms.

      Before KIP-33, deleteHorizonMs in LogCleaner is calculated based on the last modified time, which is monotonically increasing from old to new segments. With KIP-33, deleteHorizonMs is calculated based on the message timestamp, which is not necessarily monotonically increasing.

        Attachments

          Activity

            People

            • Assignee:
              becket_qin Jiangjie Qin
              Reporter:
              junrao Jun Rao
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: