[KAFKA-16378] Under tiered storage, deleting local logs does not free disk space

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.7.0
• Component/s: Tiered-Storage
• Assignee: Unassigned
• Reporter: Jianbin Chen
• Created: 15/Mar/24 01:34
• Updated: 18/Apr/24 12:35
• Resolved: 19/Mar/24 08:39

Description

Of course, this is an occasional phenomenon, as long as the tiered storage topic triggered the deletion of the local log action, there is always the possibility of residual file references, but these files on the local disk is already impossible to find!

I use the implementation as: Aiven-Open/tiered-storage-for-apache-kafka: RemoteStorageManager for Apache Kafka® Tiered Storage (github.com)

I also filed an issue in their community, which also contains a full description of the problem

Disk space not released · Issue #513 · Aiven-Open/tiered-storage-for-apache-kafka (github.com)

You can clearly see in this figure that the kafka log has already output the log of the operation that deleted the log, but the log is still referenced and the disk space has not been released