[MESOS-736] Support catch-up replicated log

• Type: Story
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.17.0
• Component/s: replicated log
• Labels: twitter
• Assignee: Jie Yu
• Reporter: Jie Yu
• Created: 14/Oct/13 18:45
• Updated: 12/Feb/14 23:23
• Resolved: 17/Jan/14 01:02

Description

When a replica joins Paxos with an empty log, we don't allow this replica to fully participate in Paxos immediately. Instead, this replica is treated as a non-voting member, meaning that it will not reply to any requests from other replicas. It simply learns those log entries that have been agreed on and tries to catch up the leader. When the catch-up process is done, we re-admit this replica to Paxos and allow it to vote.

If we have a disk failure and want to swap a master machine, we can simply start the scheduler on a new machine with an empty log (and that's it).

Issue Links

• blocks: MESOS-962 A developer guide on the catch-up log for the Mesos developer community
• blocks: MESOS-961 A blog post on the catch-up log for the Mesos user community
• depends upon: MESOS-807 Discard is not propagated in process::dispatch.
• depends upon: MESOS-806 Allowing converting from an Owned<T> to a Shared<T>.
• is depended upon by: MESOS-770 Rate control and randomization of Replicated Log catching-up
• is related to: MESOS-993 Performance issue during replicated log catch-up when the initial log position is large
• relates to: MESOS-683 Support reconfigurable replicated log