Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3143

inconsistent state in ZK when all replicas are dead

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: controller
    • Labels:

      Description

      This issue can be recreated in the following steps.

      1. Start 3 brokers, 1, 2 and 3.
      2. Create a topic with a single partition and 2 replicas, say on broker 1 and 2.
      If we stop both replicas 1 and 2, depending on where the controller is, the leader and isr stored in ZK in the end are different.

      If the controller is on broker 3, what's stored in ZK will be -1 for leader and an empty set for ISR.

      On the other hand, if the controller is on broker 2 and we stop broker 1 followed by broker 2, what's stored in ZK will be 2 for leader and 2 for ISR.

      The issue is that in the first case, the controller will call ReplicaStateMachine to transition to OfflineReplica, which will change the leader and isr. However, in the second case, the controller fails over, but we don't transition ReplicaStateMachine to OfflineReplica during controller initialization.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ijuma Ismael Juma
                Reporter:
                junrao Jun Rao
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: