Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8002

Replica reassignment to new log dir may not complete if future and current replicas segment files have different base offsets

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.1.0, 1.1.1, 2.0.0, 2.0.1, 2.1.0, 2.1.1
    • 1.1.2, 2.2.0, 2.0.2, 2.1.2
    • core
    • None

    Description

      Once future replica fetches log end offset, the intended logic is to finish the move (and rename the future dir to current replica dir, etc). However, the check in Partition.maybeReplaceCurrentWithFutureReplica compares  the whole LogOffsetMetadata vs. log end offset. The resulting behavior is that the re-assignment will not finish for topic partitions that were cleaned/ compacted such that base offset of the last segment is different for the current and future replica. 

      The proposed fix is to compare only log end offsets of the current and future replica.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            bob-barrett Bob Barrett
            apovzner Anna Povzner
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment