Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-12576

Renaming log segment should not result in a copy

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: log, log cleaner
    • Labels:
      None

      Description

      FileRecords::renameTo uses Utils::atomicMoveWithFallback to rename the backing File. The implementation for Utils::atomicMoveWithFallback will perform a non-atomic move if the atomic move fails. This can result in the content of the file being copied.

      Log uses FileRecords::renameTo indirectly through LogSegment::changeFileSuffixes while holding the Log lock to delete segment and to replace clean segments.

      This can cause the Kafka process to perform an expensive copy operation while holding a highly contended lock.

      Given how Utils::atomicMoveWithFallback works there is also an issue where File file and FileChannel channel point to different file. Meaning writes to channel are not reflected in the file represented by file.

      FileRecors::renameTo should be change to use only an atomic move and to fail if it cannot.

        Attachments

          Activity

            People

            • Assignee:
              jagsancio Jose Armando Garcia Sancio
              Reporter:
              jagsancio Jose Armando Garcia Sancio
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: