Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-12576

Renaming log segment should not result in a copy

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • log, log cleaner
    • None

    Description

      FileRecords::renameTo uses Utils::atomicMoveWithFallback to rename the backing File. The implementation for Utils::atomicMoveWithFallback will perform a non-atomic move if the atomic move fails. This can result in the content of the file being copied.

      Log uses FileRecords::renameTo indirectly through LogSegment::changeFileSuffixes while holding the Log lock to delete segment and to replace clean segments.

      This can cause the Kafka process to perform an expensive copy operation while holding a highly contended lock.

      Given how Utils::atomicMoveWithFallback works there is also an issue where File file and FileChannel channel point to different file. Meaning writes to channel are not reflected in the file represented by file.

      FileRecors::renameTo should be change to use only an atomic move and to fail if it cannot.

      Attachments

        Activity

          People

            jagsancio Jose Armando Garcia Sancio
            jagsancio Jose Armando Garcia Sancio
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: