Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2799

WakupException thrown in the followup poll() could lead to data loss

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0.1
    • Component/s: None
    • Labels:
      None

      Description

      The common pattern of the new consumer:

      try {
         records = consumer.poll();
         // process records
      } catch (WakeupException) {
         consumer.close()
      }
      

      in which the close() can commit offsets. But since in the poll() call, we do the following order:

      1) trigger client.poll().
      2) possibly update consumed position if there are some data from fetch response.
      3) before return the records, possibly trigger another client.poll()

      And if wakeup exception is thrown in 3) it will lead to not-returned messages to be committed hence data loss.

        Attachments

          Activity

            People

            • Assignee:
              guozhang Guozhang Wang
              Reporter:
              guozhang Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: