[FLUME-1516] FileChannel Write Dual Checkpoints to avoid replays

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.4.0
• Component/s: Channel, File Channel
• Assignee: Hari Shreedharan
• Reporter: Brock Noland
• Created: 27/Aug/12 19:08
• Updated: 07/Apr/13 03:43
• Resolved: 05/Apr/13 18:28

Description

Per the LFS paper (http://www.cs.berkeley.edu/~brewer/cs262/LFS.pdf) we can write two checkpoints to avoid replaying the logs in the case we crash/shutdown while writing a checkpoint.

Section 4:

"In order to handle a crash during a checkpoint operation there are actually two checkpoint regions, and checkpoint operations alternate between them. The checkpoint time is in the last block of the checkpoint so if the checkpoint fails the time will not be updated. During reboot, the system reads both checkpoint regions and uses the one with the most recent time."

Issue Links

• is related to: FLUME-1968 FileChannel new format while being backwards compatible
• links to: Review