Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-23286

Improve MTTR: Split WAL to HFile

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0-alpha-1, 2.3.0
    • 3.0.0-alpha-1, 2.3.0
    • MTTR
    • None
    • Hide
      Add a new feature to improve MTTR which have 3 steps to failover:
      1. Read WAL and write HFile to region’s column family’s recovered.hfiles directory.
      2. Open region.
      3. Bulkload the recovered.hfiles for every column family.

      Compared to DLS(distributed log split), this feature will reduce region open time significantly.

      Config hbase.wal.split.to.hfile to true to enable this featue.
      Show
      Add a new feature to improve MTTR which have 3 steps to failover: 1. Read WAL and write HFile to region’s column family’s recovered.hfiles directory. 2. Open region. 3. Bulkload the recovered.hfiles for every column family. Compared to DLS(distributed log split), this feature will reduce region open time significantly. Config hbase.wal.split.to.hfile to true to enable this featue.

    Description

      After HBASE-20724, the compaction event marker is not used anymore when failover. So our new proposal is split WAL to HFile to imporve MTTR. It has 3 steps:

      1. Read WAL and write HFile to region’s column family’s recovered.hfiles directory.
      2. Open region.
      3. Bulkload the recovered.hfiles for every column family.

      The design doc was attathed by a google doc. Any suggestions are welcomed.

      Attachments

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              zghao Guanghao Zhang
              zghao Guanghao Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: