[HUDI-6351] Long standing replace inflights post completion can be archived before cleaning for KEEP_LATEST_FILE_VERSIONS cleaner policy.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: cleaning, table-service
• Assignee: Unassigned
• Reporter: Surya Prasanna Yalla
• Created: 10/Jun/23 19:23
• Updated: 02/Sep/24 05:57

Description

For KEEP_LATEST_FILE_VERSIONS cleaner policy, file versions are only maintained for active file groups not for replaced file groups.

Since, earliestCommitToRetain is null for KEEP_LATEST_FILE_VERSIONS policy, last clean instant can be considered as a lower bound, since the cleaner would have removed all the file groups until then. But there is a catch to this logic, while cleaner is running if there is a pending replacecommit then those files are not cleaned.