Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9876 Implement Raft Protocol for Metadata Quorum
  3. KAFKA-12460

Raft should prevent truncation below high watermark

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      Eventually we will have to come up with some approach to recover from committed data loss in the raft quorum (something akin to unclean leader election for normal partitions). For now, we would rather be stricter and fail fast rather than allowing committed data to be silently lost. Specifically, we can prevent any attempt to truncate below the high watermark since this is a clear indication of data loss. The long term thought I have in mind is to give users an --unsafe flag or something like that which can be passed at startup in order to knowingly turn off the stricter validation in order to let the quorum recover from a disaster scenario. This needs some more thought though.

      Attachments

        Issue Links

          Activity

            People

              hachikuji Jason Gustafson
              hachikuji Jason Gustafson
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: