[KAFKA-2163] Offsets manager cache should prevent stale-offset-cleanup while an offset load is in progress; otherwise we can lose consumer offsets

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Pending Closed
• Fix Version/s: 0.9.0.0
• Assignee: Joel Jacob Koshy
• Reporter: Joel Jacob Koshy
• Reviewer: Jun Rao
• Created: 02/May/15 06:45
• Updated: 08/Mar/16 16:22
• Resolved: 12/May/15 22:42

Description

When leadership of an offsets partition moves, the new leader loads offsets from that partition into the offset manager cache.
Independently, the offset manager has a periodic cleanup task for stale offsets that removes old offsets from the cache and appends tombstones for those. If the partition happens to contain much older offsets (earlier in the log) and inserts those into the cache; the cleanup task may run and see those offsets (which it deems to be stale) and proceeds to remove from the cache and append a tombstone to the end of the log. The tombstone will override the true latest offset and a subsequent offset fetch request will return no offset.
We just need to prevent the cleanup task from running during an offset load.