Kafka
  1. Kafka
  2. KAFKA-948

ISR list in LeaderAndISR path not updated for partitions when Broker (which is not leader) is down

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: controller
    • Labels:
      None

      Description

      When the broker which is the leader for a partition is down, the ISR list in the LeaderAndISR path is updated. But if the broker , which is not a leader of the partition is down, the ISR list is not getting updated. This is an issues because ISR list contains the stale entry.

      This issue I found in kafka-0.8.0-beta1-candidate1

        Activity

        Hide
        Jun Rao added a comment -

        Is this still happening in 0.8?

        Show
        Jun Rao added a comment - Is this still happening in 0.8?
        Hide
        David Lao added a comment -

        Please ignore my last comment. What I ran into was KAFKA-330 (add deleted topic issue)

        Show
        David Lao added a comment - Please ignore my last comment. What I ran into was KAFKA-330 (add deleted topic issue)
        Hide
        David Lao added a comment -

        I think I just ran into a similar issue... Here are the repro steps:

        1) create a topic with 1 partition and 1 replica, populate some data
        2) use the list command to find the broker holding the partition and take the broker down
        3) use the delete command to delete the topic and then create command to re-create the same topic (so the partition will be on other brokers)

        Now use the list command to dump the topic information, note the isr: list is blank, and the leader: is set to "none"

        Show
        David Lao added a comment - I think I just ran into a similar issue... Here are the repro steps: 1) create a topic with 1 partition and 1 replica, populate some data 2) use the list command to find the broker holding the partition and take the broker down 3) use the delete command to delete the topic and then create command to re-create the same topic (so the partition will be on other brokers) Now use the list command to dump the topic information, note the isr: list is blank, and the leader: is set to "none"
        Hide
        Jun Rao added a comment -

        Dibyendu,

        Actually, the controller doesn't need to cache the latest ISR information. The only time when the controller needs ISR is to elect a new leader. The controller will read the latest ISR from ZK and do a conditional update.

        Coud you provide more details on the issue that you are seeing?

        Show
        Jun Rao added a comment - Dibyendu, Actually, the controller doesn't need to cache the latest ISR information. The only time when the controller needs ISR is to elect a new leader. The controller will read the latest ISR from ZK and do a conditional update. Coud you provide more details on the issue that you are seeing?
        Hide
        Dibyendu Bhattacharya added a comment -

        I have tested a Fix, which seems to be working fine.

        For ReplicaStateMachine.scala, the handleStateChange method for offlineReplica case, the currLeaderIsrAndControllerEpoch.leaderAndIsr.isr contains the stale entry. A call to updateLeaderAndIsrCache() before will solve this issue. Shall I go ahead and make a pull request with this fix ?

        Dibyendu

        Show
        Dibyendu Bhattacharya added a comment - I have tested a Fix, which seems to be working fine. For ReplicaStateMachine.scala, the handleStateChange method for offlineReplica case, the currLeaderIsrAndControllerEpoch.leaderAndIsr.isr contains the stale entry. A call to updateLeaderAndIsrCache() before will solve this issue. Shall I go ahead and make a pull request with this fix ? Dibyendu

          People

          • Assignee:
            Neha Narkhede
            Reporter:
            Dibyendu Bhattacharya
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development