Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      I'd love to use Apache Kafka, but for my application data loss is not acceptable. Even at the expense of availability (i.e. I need C not A in CAP).

      I think there are two things that I need to change to get a quorum model:

      1) Make sure I set request.required.acks to 2 (for a 3 node cluster) or 3 (for a 5 node cluster) on every request, so that I can only write if a quorum is active.

      2) Prevent the behaviour where a non-ISR can become the leader if all ISRs die. I think this is as easy as tweaking core/src/main/scala/kafka/controller/PartitionLeaderSelector.scala, essentially to throw an exception around line 64 in the "data loss" case.

      I haven't yet implemented / tested this. I'd love to get some input from the Kafka-experts on whether my plan is:
      (a) correct - will this work?
      (b) complete - have I missed any cases?
      (c) recommended - is this a terrible idea

      Thanks for any pointers!

        Attachments

          Activity

            People

            • Assignee:
              nehanarkhede Neha Narkhede
              Reporter:
              justinsb Justin SB
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: