Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-7278

replaceSegments() should not call asyncDeleteSegment() for segments which have been removed from segments list

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.2, 2.0.1, 2.1.0
    • Component/s: None
    • Labels:
      None

      Description

      Currently Log.replaceSegments() will call `asyncDeleteSegment(...)` for every segment listed in the `oldSegments`. oldSegments should be constructed from Log.segments and only contain segments listed in Log.segments.

      However, Log.segments may be modified between the time oldSegments is determined to the time Log.replaceSegments() is called. If there are concurrent async deletion of the same log segment file, Log.replaceSegments() will call asyncDeleteSegment() for a segment that does not exist and Kafka server may shutdown the log directory due to NoSuchFileException.

      This is likely the root cause of KAFKA-6188.

      Given the understanding of the problem, we should be able to fix the issue by only deleting segment if the segment can be found in Log.segments.

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lindong Dong Lin
                Reporter:
                lindong Dong Lin
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: