Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10786

ReplicaAlterLogDirsThread gets stuck during the reassignment of Kafka partition

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: log
    • Labels:
      None

      Description

      Topic config:Configs for topic 'athena_8603' are leader.replication.throttled.replicas=9:7,9:6,10:8,10:7,8:6,8:5,11:9,11:8,follower.replication.throttled.replicas=9:13,10:0,8:15,11:14,retention.ms=86400000,delete.retention.ms=60000

       

      Reassignment of replica athena_8603-1-15 is still in progress

       

      When I reassigning the topic partition, I found that some tasks have been in progress, lasting more than ten hours. After investigation, it was found that ReplicaAlterLogDirsThread was running all the time and occupies a high CPU usage rate (Attachment 1).
      Check the thread information (Attachment 2) and find that the log data is being copied. Check the log directory (Attachment 3) and find that the index of the future directory is older than the original log. Is it because the configuration delete.retention.ms=60000 caused the data to be deleted while copying ? This causes the replication thread to get stuck. Is there any solution?

        Attachments

        1. attachment 1.png
          271 kB
          nick song
        2. attachment 2.png
          1.44 MB
          nick song
        3. attachment 3.png
          976 kB
          nick song

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Nick0723 nick song
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: