Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5795 Make the idempotent producer the default producer setting
  3. KAFKA-5793

Tighten up situations where OutOfOrderSequence may be returned

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.0.0
    • Fix Version/s: 1.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Details of the problem are provided here: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Exactly+Once+-+Solving+the+problem+of+spurious+OutOfOrderSequence+errors

      A quick summary follows:

      In the discussion of KIP-185: Make exactly once in order delivery per partition the default producer setting, the following point regarding the OutOfOrderSequenceException was raised:

      1. The OutOfOrderSequenceException indicates that there has been data loss on the broker.. ie. a previously acknowledged message no longer exists. For most part, this should only occur in rare situations (simultaneous power outages, multiple disk losses, software bugs resulting in data corruption, etc.).
      2. However, there is another perfectly normal scenario where data is removed: in particular, data could be deleted because it is old and crosses the retention threshold.
      Hence, if a producer remains inactive for longer than a topic's retention period, we could get an OutOfOrderSequence which is a false positive: the data is removed through valid processes, and this isn't an error.
      3. We would like to eliminate the possibility of getting spurious OutOfOrderSequenceExceptions – when you get it, it should always mean data loss and should be taken very seriously.

        Attachments

          Activity

            People

            • Assignee:
              apurva Apurva Mehta
              Reporter:
              apurva Apurva Mehta
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: