[KAFKA-727] broker can still expose uncommitted data to a consumer

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Component/s: core
• Labels: p1
• Assignee: Jay Kreps
• Reporter: Jun Rao
• Created: 23/Jan/13 05:22
• Updated: 23/Dec/14 17:57
• Resolved: 23/Jan/13 22:10

Description

Even after kafka-698 is fixed, we still see consumer clients occasionally see uncommitted data. The following is how this can happen.

1. In Log.read(), we pass in startOffset < HW and maxOffset = HW.
2. Then we call LogSegment.read(), in which we call translateOffset on the maxOffset. The offset doesn't exist and translateOffset returns null.
3. Continue in LogSegment.read(), we then call messageSet.sizeInBytes() to fetch and return the data.

What can happen is that between step 2 and step 3, a new message is appended to the log and is not committed yet. Now, we have exposed uncommitted data to the client.

Issue Links

• is duplicated by: KAFKA-725 Broker Exception: Attempt to read with a maximum offset less than start offset