Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23286

Improve MTTR: Split WAL to HFile

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha-1, 2.3.0
    • Fix Version/s: 3.0.0-alpha-1, 2.3.0
    • Component/s: MTTR
    • Labels:
      None
    • Release Note:
      Hide
      Add a new feature to improve MTTR which have 3 steps to failover:
      1. Read WAL and write HFile to region’s column family’s recovered.hfiles directory.
      2. Open region.
      3. Bulkload the recovered.hfiles for every column family.

      Compared to DLS(distributed log split), this feature will reduce region open time significantly.

      Config hbase.wal.split.to.hfile to true to enable this featue.
      Show
      Add a new feature to improve MTTR which have 3 steps to failover: 1. Read WAL and write HFile to region’s column family’s recovered.hfiles directory. 2. Open region. 3. Bulkload the recovered.hfiles for every column family. Compared to DLS(distributed log split), this feature will reduce region open time significantly. Config hbase.wal.split.to.hfile to true to enable this featue.

      Description

      After HBASE-20724, the compaction event marker is not used anymore when failover. So our new proposal is split WAL to HFile to imporve MTTR. It has 3 steps:

      1. Read WAL and write HFile to region’s column family’s recovered.hfiles directory.
      2. Open region.
      3. Bulkload the recovered.hfiles for every column family.

      The design doc was attathed by a google doc. Any suggestions are welcomed.

        Attachments

          Issue Links

          There are no Sub-Tasks for this issue.

            Activity

              People

              • Assignee:
                zghao Guanghao Zhang
                Reporter:
                zghao Guanghao Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                22 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: