Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-16841

Data loss in MOB files after cloning a snapshot and deleting that snapshot



    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: mob, snapshots
    • Labels:
    • Hadoop Flags:


      Running the following steps will probably lose MOB data when working with snapshots.
      1. Create a mob-enabled table by running create 't1',

      {NAME => 'f1', IS_MOB => true, MOB_THRESHOLD => 0}

      2. Put millions of data.
      3. Run snapshot 't1','t1_snapshot' to take a snapshot for this table t1.
      4. Run clone_snapshot 't1_snapshot','t1_cloned' to clone this snapshot.
      5. Run delete_snapshot 't1_snapshot' to delete this snapshot.
      6. Run disable 't1' and delete 't1' to delete the table.
      7. Now go to the archive directory of t1, the number of .link directories is different from the number of hfiles which means some data will be lost after the hfile cleaner runs.

      This is because, when taking a snapshot on a enabled mob table, each region flushes itself and takes a snapshot, and the mob snapshot is taken only if the current region is first region of the table. At that time, the flushing of some regions might not be finished, and some mob files are not flushed to disk yet. Eventually some mob files are not recorded in the snapshot manifest.
      To solve this, we need to take the mob snapshot at last after the snapshots on all the online and offline regions are finished in EnabledTableSnapshotHandler.


        1. HBASE-16841.patch
          9 kB
          Jingcheng Du
        2. HBASE-16841-V2.patch
          13 kB
          Jingcheng Du
        3. HBASE-16841-V3.patch
          14 kB
          Jingcheng Du
        4. HBASE-16841-V4.patch
          21 kB
          Jingcheng Du
        5. HBASE-16841-V5.patch
          21 kB
          Jingcheng Du
        6. HBASE-16841-V6.patch
          20 kB
          Jingcheng Du



            • Assignee:
              jingcheng.du@intel.com Jingcheng Du
              jingcheng.du@intel.com Jingcheng Du
            • Votes:
              2 Vote for this issue
              10 Start watching this issue


              • Created: