Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24255

GCRegionProcedure doesn't assign region from RegionServer leading to orphans

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.2.4
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      hbase 2.2.4

      hadoop 3.1.3

      Description

      We've found ourselves in a situation where parents of merged or split regions needed to be opened again on a regionserver due to having to recover from cluster meltdown (HBCK2's fixMeta kicks off GCMultipleMergedRegionsProcedure which requiters all regions to be merged to be open). Then, when a GCProcedure is kicked of to clean a parent region up by GCMultipleMergedRegionsProcedure, it ends up deleting it from hbase:meta, but doesn't unassign it from RegionServer leading for it to show up in "Orphan Regions on RegionServer" in hbck tab of HBase Master. Also, the hbase client doesn't detect that the region is closed either because it's still technically open on a regionserver (it doesn't reread hbase:meta all the time). The only way to recover from this is to restart regionserver which isn't idea as it can lead to other issues in clusters with region inconsistencies.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                timoha Andrey Elenskiy
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: