Kafka
  1. Kafka
  2. KAFKA-1373

LogCleaner assumes first dirty offset zero if there is no cleaner checkpoint

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.1
    • Fix Version/s: 0.8.1.1
    • Component/s: None
    • Labels:
      None

      Description

      If you try enabling the compaction policy on a topic that already exists and if its first segment starts with anything other than zero you will see something like this:

      java.lang.IllegalArgumentException: requirement failed: Last clean offset is 0 but segment base offset is 2722629 for log test-0.
      at scala.Predef$.require(Predef.scala:145)
      at kafka.log.Cleaner.buildOffsetMap(LogCleaner.scala:489)
      at kafka.log.Cleaner.clean(LogCleaner.scala:287)
      at kafka.log.LogCleaner$CleanerThread.cleanOrSleep(LogCleaner.scala:203)
      at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:188)
      at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)

      1. KAFKA-1373.patch
        4 kB
        Joel Koshy
      2. KAFKA-1373_2014-04-08_16:18:22.patch
        5 kB
        Joel Koshy
      3. KAFKA-1373.patch
        3 kB
        Joel Koshy

        Issue Links

          Activity

          Hide
          Joel Koshy added a comment -

          Created reviewboard https://reviews.apache.org/r/20130/
          against branch origin/trunk

          Show
          Joel Koshy added a comment - Created reviewboard https://reviews.apache.org/r/20130/ against branch origin/trunk
          Hide
          Joel Koshy added a comment -

          Updated reviewboard https://reviews.apache.org/r/20130/
          against branch origin/trunk

          Show
          Joel Koshy added a comment - Updated reviewboard https://reviews.apache.org/r/20130/ against branch origin/trunk
          Hide
          Joel Koshy added a comment -

          Updated the RB with an additional fix: KAFKA-1289 changed the compaction config from dedupe to compact. That needed to be updated in a couple of other places as well. Ideally it should have been some global constant string but this is fine for now.

          Show
          Joel Koshy added a comment - Updated the RB with an additional fix: KAFKA-1289 changed the compaction config from dedupe to compact. That needed to be updated in a couple of other places as well. Ideally it should have been some global constant string but this is fine for now.
          Hide
          Joel Koshy added a comment -

          Committed to trunk, but need to submit a separate patch for 0.8.1 branch.

          Show
          Joel Koshy added a comment - Committed to trunk, but need to submit a separate patch for 0.8.1 branch.
          Hide
          Joel Koshy added a comment -

          Created reviewboard https://reviews.apache.org/r/20227/
          against branch origin/0.8.1

          Show
          Joel Koshy added a comment - Created reviewboard https://reviews.apache.org/r/20227/ against branch origin/0.8.1
          Hide
          Joel Koshy added a comment -

          Committed to 0.8.1 also.

          Show
          Joel Koshy added a comment - Committed to 0.8.1 also.

            People

            • Assignee:
              Joel Koshy
              Reporter:
              Joel Koshy
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development