[HUDI-854] Incremental Cleaning should not revert to brute force all-partition scanning in any cases

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: cleaning
• Labels: help-requested starter
• Assignee: liwei
• Reporter: Balaji Varadarajan
• Created: 30/Apr/20 09:13
• Updated: 09/Jul/20 13:10

Description

After https://github.com/apache/incubator-hudi/pull/1576 . Incremental Cleaning would still resort to full partition scan when  no previous clean operation was done in the dataset. This ticket is to design and implement a safe solution which would avoid full scanning in all cases.