[KAFKA-15272] Fix the logic which finds candidate log segments to upload it to tiered storage

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.6.0
• Assignee: Kamal Chandraprakash
• Reporter: Kamal Chandraprakash
• Created: 29/Jul/23 08:03
• Updated: 20/Sep/23 04:02
• Resolved: 02/Aug/23 10:32

Description

In tiered storage, a segment is eligible for deletion from local disk when it gets uploaded to the remote storage.

If the topic active segment contains some messages and there are no new incoming messages, then the active segment gets rotated to passive segment after the configured log.roll.ms timeout.

 

The logic to find the candidate segment in RemoteLogManager does not include the recently rotated passive segment as eligible to upload it to remote storage so the passive segment won't be removed even after if it breaches by retention time/size. (ie) Topic won't be empty after it becomes stale.

Issue Links

• links to: GitHub Pull Request #14128