Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16452

Bound highwatermark offset to range b/w local-log-start-offset and log-end-offset

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.8.0
    • None
    • None

    Description

      The high watermark should not go below the local-log-start offset. If the high watermark is less than the local-log-start-offset, then the UnifiedLog#fetchHighWatermarkMetadata method will throw the OFFSET_OUT_OF_RANGE error when it converts the offset to metadata. Once this error happens, the followers will receive out-of-range exceptions and the producers won't be able to produce messages since the leader cannot move the high watermark.

      This issue can happen when the partition undergoes recovery due to corruption in the checkpoint file and it gets elected as leader before it gets a chance to update the HW from the previous leader.

      Attachments

        Issue Links

          is a child of

          Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15420 Kafka Tiered Storage V1

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #15634

          Web Link GitHub Pull Request #15825

          Activity

            People

              ckamal Kamal Chandraprakash
              ckamal Kamal Chandraprakash
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: