Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • hbase-11339
    • hbase-11339, 2.0.0
    • mob
    • None
    • Reviewed

    Description

      In major compaction, the mvcc of cells ( whose mvcc<=readPt) are set to 0.
      In some cases, this brings issues, for example the following scenario:

      1. We have mob enabled cf, the threshold is 5 bytes.
      2. Add a cell (r0,ts0,seqId=5,"mobValue0"), and flush it to a mob file.
      3. Add another cell (r0,ts0,seqId=10,"new"), and flush the memstore, this is not a mob cell since it's value is smaller than 5 bytes.
      4. Add the third cell (r1:ts1:seqId =15, "mobValue1"), and flush it to a mob file. Now we have two mob files.
      5. Now run a major compaction in hfiles, we got two cells left (r0:ts0:seqId=0,"new") and (r1:ts1:seqId=0,'mobValue1").
      6. Now run a mob major compaction, two mob files are merged into one. The update ref cell is bulk loaded back to hbase, they are (r0,ts0,seqId=5,"mobValue0") and (r1:ts1:seqId=0,"mobValue1").
      7. Now open a scanner, the value of r0 is mobValue0 whereas the correct value new.

      This issue is caused by the mvcc reset in compactions. We should disable it in compactions for mob-enabled columns.

      Attachments

        Activity

          jingcheng.du@intel.com Jingcheng Du added a comment -

          Hi Jonathan Hsieh, Anoop Sam John, ramkrishna.s.vasudevan, would you mind reviewing the patch. thanks a lot!

          jingcheng.du@intel.com Jingcheng Du added a comment - Hi Jonathan Hsieh , Anoop Sam John , ramkrishna.s.vasudevan , would you mind reviewing the patch. thanks a lot!

          +1 on patch.

          ram_krish ramkrishna.s.vasudevan added a comment - +1 on patch.
          jingcheng.du@intel.com Jingcheng Du added a comment -

          Hi Anoop Sam John, do you want to take a look? Thanks.

          jingcheng.du@intel.com Jingcheng Du added a comment - Hi Anoop Sam John , do you want to take a look? Thanks.
          anoop.hbase Anoop Sam John added a comment -

          mvcc can never get reset for the MOB cfs is a concern only for me. Normal stores, atleast at major compaction time we can get this reset. But for the case mentioned, can not think of any other way. If the duplicate cells can be added with same TS we will land in this issue. So +1 to commit it now.

          anoop.hbase Anoop Sam John added a comment - mvcc can never get reset for the MOB cfs is a concern only for me. Normal stores, atleast at major compaction time we can get this reset. But for the case mentioned, can not think of any other way. If the duplicate cells can be added with same TS we will land in this issue. So +1 to commit it now.
          hudson Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #6672 (See https://builds.apache.org/job/HBase-TRUNK/6672/)
          HBASE-13922 Do not reset mvcc in compactions for mob-enabled column.(Jingcheng Du) (anoopsamjohn: rev 3f062ee23668020c15f9d06a966a0978ca9373f6)

          • hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java
          • hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java
          hudson Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #6672 (See https://builds.apache.org/job/HBase-TRUNK/6672/ ) HBASE-13922 Do not reset mvcc in compactions for mob-enabled column.(Jingcheng Du) (anoopsamjohn: rev 3f062ee23668020c15f9d06a966a0978ca9373f6) hbase-server/src/main/java/org/apache/hadoop/hbase/mob/DefaultMobStoreCompactor.java hbase-server/src/test/java/org/apache/hadoop/hbase/mob/compactions/TestMobCompactor.java

          People

            jingcheng.du@intel.com Jingcheng Du Assign to me
            jingcheng.du@intel.com Jingcheng Du
            Votes:
            0 Vote for this issue
            Watchers:
            Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack