[KAFKA-4680] min.insync.replicas can be set higher than replication factor

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.10.0.1
• Component/s: config, replication
• Assignee: Paolo Moriello
• Reporter: James Cheng
• Created: 21/Jan/17 00:05
• Updated: 22/Nov/24 03:42

Description

It is possible to specify a min.insync.replicas for a topic that is higher than the replication factor of the topic. If you do this, you will not be able to produce to the topic with acks=all.

Furthermore, each produce request (including retries) to the topic will emit an ERROR level message to the broker debuglogs. If this is not noticed quickly enough, it can cause the debuglogs to balloon.

We actually hosed one of our Kafka clusters because of this. A topic got configured with min.insync.replicas > replication factor. It had partitions on all brokers of our cluster. The broker logs ballooned and filled up the disks. We run these clusters on CoreOS, and CoreOS's etcd database got corrupted. (Kafka didn't get corrupted, tho).

I think Kafka should do validation when someone tries to change a topic to min.insync.replicas > replication factor, and reject the change.

This would presumably affect kafka-topics.sh, kafka-configs.sh, as well as the CreateTopics operation that came in KIP-4.

Issue Links

• is duplicated by: KAFKA-8695 Metrics UnderReplicated and UnderMinIsr are diverging when configuration is inconsistent
• links to: GitHub Pull Request #8225