[OAK-5278] Improved compaction estimator

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Later
• Component/s: segment-tar
• Labels: gc operations
• Epic Link: TarMK GC scalability
• Assignee: Unassigned
• Reporter: Michael Dürig
• Created: 12/Dec/16 15:49
• Updated: 19/Jun/17 10:27
• Resolved: 19/Jun/17 10:27

Description

OAK-4293 introduced a new approach for estimating whether we actually want to run or skip a gc cycle. That approach is purely based on the absolute growth of the repository's on disk footprint.

I think this can be further refined as with the GCJournal we can effectively extrapolate the amount of garbage at a given point in time given the history of previous gc cycles. E.g. let S_n be the size of the repository and G_n the percentage of garbage right before the n-th gc cycle. We can then linearly extrapolate the garbage G_n+1 for the n+1-the gc cycle along the repository sizes:

G_n+1 = G_n * (S_k+1 - S_k)/(S_k - S_k-1)

Issue Links

• relates to: OAK-4689 Add information about amount of data vs. waste to oak-run