[KAFKA-7385] Log cleaner crashes when empty batches are retained with idempotent or transactional producers

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.1
• Assignee: Dhruvil Shah
• Reporter: Dhruvil Shah
• Created: 07/Sep/18 15:31
• Updated: 26/Oct/18 15:43
• Resolved: 09/Sep/18 01:07

Description

During log compaction, we retain an empty header if the batch contains the last sequence number for a particular producer. When such headers are the only messages retained, we do not update state such as `maxOffset` in `MemoryRecords#filterTo` causing us to append these into the cleaned segment with `largestOffset` = -1. This throws a `LogSegmentOffsetOverflowException` for a segment that does not actually have an overflow. When we attempt to split the segment, the log cleaner dies.

Issue Links

• links to: GitHub Pull Request #5623