Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20727

Persist FlushedSequenceId to speed up WAL split after cluster restart

    Details

    • Type: New Feature
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.0.0
    • Fix Version/s: 3.0.0
    • Component/s: None
    • Labels:
      None

      Description

      We use flushedSequenceIdByRegion and storeFlushedSequenceIdsByRegion in ServerManager to record the latest flushed seqids of regions and stores. So during log split, we can use seqids stored in those maps to filter out the edits which do not need to be replayed. But, those maps are not persisted. After cluster restart or master restart, info of flushed seqids are all lost.
      Here I offer a way to persist those info to HDFS, even if master restart, we can still use those info to filter WAL edits and then to speed up replay.

        Attachments

        1. HBASE-20727.patch
          17 kB
          Allan Yang
        2. HBASE-20727.005.patch
          17 kB
          Allan Yang
        3. HBASE-20727.004.patch
          17 kB
          Allan Yang
        4. HBASE-20727.003.patch
          17 kB
          Allan Yang
        5. HBASE-20727.002.patch
          19 kB
          Allan Yang

          Activity

            People

            • Assignee:
              allan163 Allan Yang
              Reporter:
              allan163 Allan Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: