Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4680

min.insync.replicas can be set higher than replication factor

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.10.0.1
    • Fix Version/s: None
    • Component/s: config, replication
    • Labels:
      None

      Description

      It is possible to specify a min.insync.replicas for a topic that is higher than the replication factor of the topic. If you do this, you will not be able to produce to the topic with acks=all.

      Furthermore, each produce request (including retries) to the topic will emit an ERROR level message to the broker debuglogs. If this is not noticed quickly enough, it can cause the debuglogs to balloon.

      We actually hosed one of our Kafka clusters because of this. A topic got configured with min.insync.replicas > replication factor. It had partitions on all brokers of our cluster. The broker logs ballooned and filled up the disks. We run these clusters on CoreOS, and CoreOS's etcd database got corrupted. (Kafka didn't get corrupted, tho).

      I think Kafka should do validation when someone tries to change a topic to min.insync.replicas > replication factor, and reject the change.

      This would presumably affect kafka-topics.sh, kafka-configs.sh, as well as the CreateTopics operation that came in KIP-4.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                paolomoriello Paolo Moriello
                Reporter:
                wushujames James Cheng
              • Votes:
                1 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated: