[KAFKA-4680] min.insync.replicas can be set higher than replication factor - ASF JIRA

Agile Board

Attach files

Attach Screenshot

Add vote

Voters

Stop watching

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 0.10.0.1
Fix Version/s: None
Component/s: config, replication
Labels:
None

Description

It is possible to specify a min.insync.replicas for a topic that is higher than the replication factor of the topic. If you do this, you will not be able to produce to the topic with acks=all.

Furthermore, each produce request (including retries) to the topic will emit an ERROR level message to the broker debuglogs. If this is not noticed quickly enough, it can cause the debuglogs to balloon.

We actually hosed one of our Kafka clusters because of this. A topic got configured with min.insync.replicas > replication factor. It had partitions on all brokers of our cluster. The broker logs ballooned and filled up the disks. We run these clusters on CoreOS, and CoreOS's etcd database got corrupted. (Kafka didn't get corrupted, tho).

I think Kafka should do validation when someone tries to change a topic to min.insync.replicas > replication factor, and reject the change.

This would presumably affect kafka-topics.sh, kafka-configs.sh, as well as the CreateTopics operation that came in KIP-4.