Affects Version/s: None
Fix Version/s: None
I'd love to use Apache Kafka, but for my application data loss is not acceptable. Even at the expense of availability (i.e. I need C not A in CAP).
I think there are two things that I need to change to get a quorum model:
1) Make sure I set request.required.acks to 2 (for a 3 node cluster) or 3 (for a 5 node cluster) on every request, so that I can only write if a quorum is active.
2) Prevent the behaviour where a non-ISR can become the leader if all ISRs die. I think this is as easy as tweaking core/src/main/scala/kafka/controller/PartitionLeaderSelector.scala, essentially to throw an exception around line 64 in the "data loss" case.
I haven't yet implemented / tested this. I'd love to get some input from the Kafka-experts on whether my plan is:
(a) correct - will this work?
(b) complete - have I missed any cases?
(c) recommended - is this a terrible idea
Thanks for any pointers!
|Field||Original Value||New Value|
|Assignee||Neha Narkhede [ nehanarkhede ]|
|Status||Open [ 1 ]||Resolved [ 5 ]|
|Resolution||Fixed [ 1 ]|
|Workflow||no-reopen-closed, patch-avail [ 12814578 ]||Apache Kafka Workflow [ 13051805 ]|
|Workflow||Apache Kafka Workflow [ 13051805 ]||no-reopen-closed, patch-avail [ 13054728 ]|
|Transition||Time In Source Status||Execution Times||Last Executer||Last Execution Date|
|517d 22h 24m||1||Jay Kreps||07/Feb/15 23:17|