Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16578

Mob data loss after mob compaction and normal compaction

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: mob
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      StoreFileScanners on MOB cells rely on the scannerOrder to find the latest cells after mob compaction. The value of scannerOrder is assigned by the order of maxSeqId of StoreFile, and this maxSeqId is valued only after the reader of the StoreFile is created.
      In Compactor.compact, the compacted store files are cloned and their readers are not created. And in StoreFileScanner.getScannersForStoreFiles the StoreFiles are sorted before the readers are created and at that time the maxSeqId for each file is -1 (the default value). This will lead to a chaos in scanners in the following normal compaction. Some older cells might be chosen during the normal compaction.
      We need to create readers either before the sorting in the method StoreFileScanner.getScannersForStoreFiles, or create readers just after the store files are cloned in Compactor.compact.

        Attachments

        1. TestMobCompaction.java
          8 kB
          Hua Xiang
        2. TestMobCompaction.java
          7 kB
          Hua Xiang
        3. HBASE-16578.patch
          9 kB
          Jingcheng Du
        4. HBASE-16578-V2.patch
          9 kB
          Jingcheng Du
        5. HBASE-16578-V3.patch
          10 kB
          Jingcheng Du

          Activity

            People

            • Assignee:
              jingcheng.du@intel.com Jingcheng Du
              Reporter:
              huaxiang Hua Xiang
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: