[MESOS-3180] Replicated log (registry) may bloat under some conditions

• Type: Improvement
• Status: Accepted
• Priority: Major
• Resolution: Unresolved
• Component/s: replicated log
• Labels: mesosphere tech-debt
• Assignee: Unassigned
• Reporter: Joseph Wu
• Created: 30/Jul/15 22:07
• Updated: 26/Nov/18 13:36

Description

This issue is to track potential problems with the replicated log if it stores diffs without proper compaction/truncation.

As part of the Maintenance Primitives and Quota, the replicated registry will be augmented with additional fields, which may be updates relatively independently. If the replicated log stores diffs, it may store a series of changes:

S = info on slaves
M = maintenance schedule
[X] = a diff containing an update to some part of the registry

Registry: [ ] -> [S1] -> [M1] -> [M2] -> [M3] -> [M4] -> ... -> [S2]

Between S1 and S2, multiple M's are stored. As each M is stored, the log cannot truncate the log because the earliest instance of S is at the beginning. Only once the S2 is reached, does the log get truncated and recover the space used.

Issue Links

• relates to: MESOS-6267 Mesos master should periodically compact the replicated log