[KAFKA-12576] Renaming log segment should not result in a copy

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: log, log cleaner
• Assignee: Jose Armando Garcia Sancio
• Reporter: Jose Armando Garcia Sancio
• Created: 30/Mar/21 01:11
• Updated: 30/Mar/21 01:48

Description

FileRecords::renameTo uses Utils::atomicMoveWithFallback to rename the backing File. The implementation for Utils::atomicMoveWithFallback will perform a non-atomic move if the atomic move fails. This can result in the content of the file being copied.

Log uses FileRecords::renameTo indirectly through LogSegment::changeFileSuffixes while holding the Log lock to delete segment and to replace clean segments.

This can cause the Kafka process to perform an expensive copy operation while holding a highly contended lock.

Given how Utils::atomicMoveWithFallback works there is also an issue where File file and FileChannel channel point to different file. Meaning writes to channel are not reflected in the file represented by file.

FileRecors::renameTo should be change to use only an atomic move and to fail if it cannot.