Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-1516

FileChannel Write Dual Checkpoints to avoid replays

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.4.0
    • Channel, File Channel
    • None

    Description

      Per the LFS paper (http://www.cs.berkeley.edu/~brewer/cs262/LFS.pdf) we can write two checkpoints to avoid replaying the logs in the case we crash/shutdown while writing a checkpoint.

      Section 4:

      "In order to handle a crash during a checkpoint operation there are actually two checkpoint regions, and checkpoint operations alternate between them. The checkpoint time is in the last block of the checkpoint so if the checkpoint fails the time will not be updated. During reboot, the system reads both checkpoint regions and uses the one with the most recent time."

      Attachments

        1. DualCheckpoints.pdf
          96 kB
          Hari Shreedharan
        2. DualCheckpointsv2.pdf
          106 kB
          Hari Shreedharan
        3. DualCheckpointsv3.pdf
          79 kB
          Hari Shreedharan
        4. FLUME-1516.patch
          44 kB
          Hari Shreedharan
        5. FLUME-1516-1.patch
          44 kB
          Hari Shreedharan
        6. FLUME-1516-2.patch
          70 kB
          Hari Shreedharan
        7. FLUME-1516-3.patch
          76 kB
          Hari Shreedharan
        8. FLUME-1516-4.patch
          76 kB
          Hari Shreedharan
        9. FLUME-1516-5.patch
          83 kB
          Hari Shreedharan
        10. FLUME-1516-6.patch
          83 kB
          Hari Shreedharan
        11. FLUME-1516-7.patch
          92 kB
          Hari Shreedharan
        12. FLUME-1516-8.patch
          97 kB
          Hari Shreedharan

        Issue Links

          Activity

            People

              hshreedharan Hari Shreedharan
              brocknoland Brock Noland
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: