Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1516

FileChannel Write Dual Checkpoints to avoid replays


    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.4.0
    • Component/s: Channel, File Channel
    • Labels:


      Per the LFS paper (http://www.cs.berkeley.edu/~brewer/cs262/LFS.pdf) we can write two checkpoints to avoid replaying the logs in the case we crash/shutdown while writing a checkpoint.

      Section 4:

      "In order to handle a crash during a checkpoint operation there are actually two checkpoint regions, and checkpoint operations alternate between them. The checkpoint time is in the last block of the checkpoint so if the checkpoint fails the time will not be updated. During reboot, the system reads both checkpoint regions and uses the one with the most recent time."


        1. DualCheckpoints.pdf
          96 kB
          Hari Shreedharan
        2. DualCheckpointsv2.pdf
          106 kB
          Hari Shreedharan
        3. DualCheckpointsv3.pdf
          79 kB
          Hari Shreedharan
        4. FLUME-1516.patch
          44 kB
          Hari Shreedharan
        5. FLUME-1516-1.patch
          44 kB
          Hari Shreedharan
        6. FLUME-1516-2.patch
          70 kB
          Hari Shreedharan
        7. FLUME-1516-3.patch
          76 kB
          Hari Shreedharan
        8. FLUME-1516-4.patch
          76 kB
          Hari Shreedharan
        9. FLUME-1516-5.patch
          83 kB
          Hari Shreedharan
        10. FLUME-1516-6.patch
          83 kB
          Hari Shreedharan
        11. FLUME-1516-7.patch
          92 kB
          Hari Shreedharan
        12. FLUME-1516-8.patch
          97 kB
          Hari Shreedharan

          Issue Links



              • Assignee:
                hshreedharan Hari Shreedharan
                brocknoland Brock Noland
              • Votes:
                0 Vote for this issue
                5 Start watching this issue


                • Created: