[KAFKA-6183] Broker should send OffsetCommitResponse only after it has written offset to cache

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Dong Lin
• Reporter: Dong Lin
• Created: 07/Nov/17 19:40
• Updated: 07/Nov/17 19:40

Description

Currently broker sends OffsetCommitResponse to client before it writes committed offset to disk and cache. Thus client does not have read-after-write semantics when committing and reading offset. The following sequence of events may happen:

• Client sends offset commit request to broker.
• Broker sends offset commit response back.
• Client sends offset fetch request to broker.
• Broker returns an empty offset fetch response to client.
• Broker writes the committed offset to disk and cache.

Broker should return OffsetCommitResponse after it has written committed offset to disk and memory, similar to the approach that broker returns ProduceResponse after it has written data to disk. Note that the data does not have to be flushed to disk. This change makes offset commit semantics easier to use without incurring cost on the broker.