Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20933

multiple splits may result into forever uncleaned split region

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Incase of multiple subsequent split and with an open handle on old reference file, it may result into split region which can never be cleaned

       So Here are two issues.

      1. Region is getting split even when it has reference to its parent
      2. Region is going offline/in archive mode even when there are reference pending in store

      Repro Steps

      1. Region split (P)
      2. Before major compaction starts after split, open a handle on store file on new region (DA & DB)
      3. Let compaction completes on DA, (Here compaction will not clear reference store files as it is opened)
      4. Split new region (DA) again ( shouldSplit will return true as before compaction even does the cleanup, it removes the compacted files and reference in-memory list)
      5. Now CatalogJanitor will not remove this region as it has store references, majorCompaction/CompactedHFilesDischarger will not do the cleanup as it only looks at only online regions
      6.  After above steps region-DA which is offline will always be in split regions and never getting cleaned up.

      We found that catalog janitor is also not able to clean regions which are offline(split parent) because it has reference of the daughter of it's parent which is not getting cleaned up. This is causing lot of store files not getting cleaned causing more space in local index store and lot of split lingering regions.

      Unit test repro the scenario has been attached.

      Fix can be in CompactedHFilesDischarger or catalogJanitor to handle such cases. Even if such region exists which are offline and are split region. They should be able to clean t hem selves

        Attachments

        1. HBASE-20933.patch
          7 kB
          Vishal Khandelwal
        2. Test123.java
          7 kB
          Vishal Khandelwal

          Issue Links

            Activity

              People

              • Assignee:
                vishk Vishal Khandelwal
                Reporter:
                vishk Vishal Khandelwal
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: