Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-11315

Keeping MVCC for configurable longer time

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.99.0
    • 0.99.0
    • None
    • None
    • Reviewed
    • Hide
      After this patch, MVCC value of KVs will be kept for a configurable period "hbase.hstore.compaction.keep.mvcc.period"(default value is 5 days). The minimum setting of "hbase.hstore.compaction.keep.mvcc.period" is also set to 5 days and any user setting less than the min setting will be ignored.
      Show
      After this patch, MVCC value of KVs will be kept for a configurable period "hbase.hstore.compaction.keep.mvcc.period"(default value is 5 days). The minimum setting of "hbase.hstore.compaction.keep.mvcc.period" is also set to 5 days and any user setting less than the min setting will be ignored.

    Description

      After hbase-8763, we need keep mvcc number longer in hfile so that it can be used to order changes during writes. For example, the known put,delete,put,... scenario, cross region server scan, out of order puts(in recovery case).

      Current thinking is that we make the retention period configurable(below we're using 1 day to explain). During major compaction, we check hfile's creation time if a hfile creation time is older than 1 day then all mvcc of KVs in that hfile will be removed. If a hfile is created within 1 day, then all mvccs of KVs in that hfile will be kept.

      In case there are time clock skew, we can firstly sort hfiles based on its seqId in ascending order and find the first hfile's creation time stamp less than 1 day. Then mvcc of all hfiles before the found file will be removed during compaction.

      Attachments

        1. hbase-11315-v1.patch
          64 kB
          Jeffrey Zhong
        2. hbase-11315.patch
          12 kB
          Jeffrey Zhong

        Issue Links

          Activity

            People

              jeffreyz Jeffrey Zhong
              jeffreyz Jeffrey Zhong
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: