[KAFKA-1641] Log cleaner exits if last cleaned offset is lower than earliest offset

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.1.1
• Fix Version/s: 0.9.0.0
• Assignee: Guozhang Wang
• Reporter: Joel Jacob Koshy
• Created: 18/Sep/14 23:05
• Updated: 20/Sep/19 01:21
• Resolved: 23/Oct/14 22:43

Description

Encountered this recently: the log cleaner exited a while ago (I think because the topic had compressed messages). That issue was subsequently addressed by having the producer only send uncompressed. However, on a subsequent restart of the broker we see this:

In this scenario I think it is reasonable to just emit a warning and have the cleaner round up its first dirty offset to the base offset of the first segment.

[kafka-server] [] [kafka-log-cleaner-thread-0], Error due to 
java.lang.IllegalArgumentException: requirement failed: Last clean offset is 54770438 but segment base offset is 382844024 for log testtopic-0.
        at scala.Predef$.require(Predef.scala:145)
        at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:491)
        at kafka.log.Cleaner.clean(LogCleaner.scala:288)
        at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:202)
        at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:187)
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)