Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1516

FileChannel Write Dual Checkpoints to avoid replays

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.4.0
    • Component/s: Channel, File Channel
    • Labels:
      None

      Description

      Per the LFS paper (http://www.cs.berkeley.edu/~brewer/cs262/LFS.pdf) we can write two checkpoints to avoid replaying the logs in the case we crash/shutdown while writing a checkpoint.

      Section 4:

      "In order to handle a crash during a checkpoint operation there are actually two checkpoint regions, and checkpoint operations alternate between them. The checkpoint time is in the last block of the checkpoint so if the checkpoint fails the time will not be updated. During reboot, the system reads both checkpoint regions and uses the one with the most recent time."

        Attachments

        1. DualCheckpoints.pdf
          96 kB
          Hari Shreedharan
        2. DualCheckpointsv2.pdf
          106 kB
          Hari Shreedharan
        3. DualCheckpointsv3.pdf
          79 kB
          Hari Shreedharan
        4. FLUME-1516.patch
          44 kB
          Hari Shreedharan
        5. FLUME-1516-1.patch
          44 kB
          Hari Shreedharan
        6. FLUME-1516-2.patch
          70 kB
          Hari Shreedharan
        7. FLUME-1516-3.patch
          76 kB
          Hari Shreedharan
        8. FLUME-1516-4.patch
          76 kB
          Hari Shreedharan
        9. FLUME-1516-5.patch
          83 kB
          Hari Shreedharan
        10. FLUME-1516-6.patch
          83 kB
          Hari Shreedharan
        11. FLUME-1516-7.patch
          92 kB
          Hari Shreedharan
        12. FLUME-1516-8.patch
          97 kB
          Hari Shreedharan

          Issue Links

            Activity

              People

              • Assignee:
                hshreedharan Hari Shreedharan
                Reporter:
                brocknoland Brock Noland
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: