[OAK-5790] Chronologically rebase checkpoints on top of each other during compaction

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.3, 1.8.0
• Component/s: segment-tar
• Labels: compaction gc performance
• Epic Link: TarMK GC scalability
• Assignee: Michael Dürig
• Reporter: Michael Dürig
• Created: 23/Feb/17 16:00
• Updated: 06/Jul/17 15:34
• Resolved: 27/Jun/17 16:23

Description

Currently the compactor does just a rewrite of the super root node without any special handling of the checkpoints. It just relies on the node de-duplication cache to avoid fully exploding the checkpoints.
I think this can be improved by subsequently rebasing checkpoints on top of each other during compaction. (Very much like checkpoints are handled in migration).

Issue Links

• blocks: OAK-6399 Re-establish full offline compaction functionality
• depends upon: OAK-5952 Wrong eviction count reported by PriorityCache statistics
• depends upon: OAK-5951 Enable Metrics for SegmentCompactionIT test
• depends upon: OAK-5953 PriorityCache statistics should support load exception count
• depends upon: OAK-5954 Unify and simplify the deduplication caches
• depends upon: OAK-5967 Null instances for SegmentId and RecordId
• is blocked by: OAK-6395 Refactor monitoring of deduplication caches