[KUDU-1625] Schedule compaction on rowsets with high percentage of deleted data

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.0.0
• Component/s: tablet
• Assignee: Unassigned
• Reporter: Todd Lipcon
• Created: 16/Sep/16 22:07
• Updated: 13/Jun/23 05:43

Description

Although with KUDU-236 we can now remove rows that were deleted prior to the ancient history mark, we don't actively schedule compactions based on deleted rows. So, if for example we have a fully compacted table and issue a DELETE for every row, the data size actually does not change, because no compactions are triggered.

We need some way to notice the fact that the ratio of deletes to rows is high and decide to compact those rowsets.

Issue Links

• is duplicated by: KUDU-1979 Row GC maintenance task
• is related to: KUDU-3367 Delta file with full of delete op can not be schedule to compact
• is related to: KUDU-1400 Improve rowset compaction policy to consider merging small DRSs