[KAFKA-2799] WakupException thrown in the followup poll() could lead to data loss

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.9.0.1
• Assignee: Guozhang Wang
• Reporter: Guozhang Wang
• Created: 10/Nov/15 18:40
• Updated: 25/Nov/15 20:00
• Resolved: 10/Nov/15 22:49

Description

The common pattern of the new consumer:

try {
   records = consumer.poll();
   // process records
} catch (WakeupException) {
   consumer.close()
}

in which the close() can commit offsets. But since in the poll() call, we do the following order:

1) trigger client.poll().
2) possibly update consumed position if there are some data from fetch response.
3) before return the records, possibly trigger another client.poll()

And if wakeup exception is thrown in 3) it will lead to not-returned messages to be committed hence data loss.