Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8739

rejoining broker fails to sanity check existing log segments

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.3.0
    • Fix Version/s: None
    • Component/s: replication
    • Labels:
      None

      Description

      kafka claims it can be used as a storage. But following scenario proves other wise.

      1. Consider a topic with single partition, repl-factor 2, with two brokers, say A and B.... with A is the leader.
      2. Broker B fails due to sector errors. Sysadmin fixes the issues and brings it up again after a few minutes. A few log segments are lost/corrupted.
      3. Broker B catches up with missed out msgs by fetching them from the leader A, but does not realize the issue with earlier log segments.
      4. Broker A fails, B becomes the leader.
      5. A new consumer requests msgs from the beginning. Broker B fails to deliver msgs belonging to corrupted log segments.

      Suggested solution

      A broker, immediately after coming up, should go through a sanity check, e.g. CRC check of its log segments. Any corrupted/lost, should be refetched from the leader.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              sanjivmarathe sanjiv marathe
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: