[HBASE-18792] hbase-2 needs to defend against hbck operations

• Type: Task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Component/s: hbck
• Hadoop Flags: Reviewed

Release Note

As of HBase version 2.0, the hbck tool is significantly changed. In general, all Read-Only options are supported and can be be used safely. Most -fix/ -repair options are NOT supported. Please see usage below for details on which options are not supported:


Usage: fsck [opts] {only tables}
 where [opts] are:
   -help Display help options (this)
   -details Display full report of all regions.
   -timelag <timeInSeconds> Process only regions that have not experienced any metadata updates in the last <timeInSeconds> seconds.
   -sleepBeforeRerun <timeInSeconds> Sleep this many seconds before checking if the fix worked if run with -fix
   -summary Print only summary of the tables and status.
   -metaonly Only check the state of the hbase:meta table.
   -sidelineDir <hdfs://> HDFS path to backup existing meta.
   -boundaries Verify that regions boundaries are the same between META and store files.
   -exclusive Abort if another hbck is exclusive or fixing.

  Datafile Repair options: (expert features, use with caution!)
   -checkCorruptHFiles Check all Hfiles by opening them to make sure they are valid
   -sidelineCorruptHFiles Quarantine corrupted HFiles. implies -checkCorruptHFiles

 Replication options
   -fixReplication Deletes replication queues for removed peers

  Metadata Repair options supported as of version 2.0: (expert features, use with caution!)
   -fixVersionFile Try to fix missing hbase.version file in hdfs.
   -fixReferenceFiles Try to offline lingering reference store files
   -fixHFileLinks Try to offline lingering HFileLinks
   -noHdfsChecking Don't load/check region info from HDFS. Assumes hbase:meta region info is good. Won't check/fix any HDFS issue, e.g. hole, orphan, or overlap
   -ignorePreCheckPermission ignore filesystem permission pre-check

NOTE: Following options are NOT supported as of HBase version 2.0+.

  UNSUPPORTED Metadata Repair options: (expert features, use with caution!)
   -fix Try to fix region assignments. This is for backwards compatiblity
   -fixAssignments Try to fix region assignments. Replaces the old -fix
   -fixMeta Try to fix meta problems. This assumes HDFS region info is good.
   -fixHdfsHoles Try to fix region holes in hdfs.
   -fixHdfsOrphans Try to fix region dirs with no .regioninfo file in hdfs
   -fixTableOrphans Try to fix table dirs with no .tableinfo file in hdfs (online mode only)
   -fixHdfsOverlaps Try to fix region overlaps in hdfs.
   -maxMerge <n> When fixing region overlaps, allow at most <n> regions to merge. (n=5 by default)
   -sidelineBigOverlaps When fixing region overlaps, allow to sideline big overlaps
   -maxOverlapsToSideline <n> When fixing region overlaps, allow at most <n> regions to sideline per group. (n=2 by default)
   -fixSplitParents Try to force offline split parents to be online.
   -removeParents Try to offline and sideline lingering parents and keep daughter regions.
   -fixEmptyMetaCells Try to fix hbase:meta entries not referencing any region (empty REGIONINFO_QUALIFIER rows)

  UNSUPPORTED Metadata Repair shortcuts
   -repair Shortcut for -fixAssignments -fixMeta -fixHdfsHoles -fixHdfsOrphans -fixHdfsOverlaps -fixVersionFile -sidelineBigOverlaps -fixReferenceFiles-fixHFileLinks
   -repairHoles Shortcut for -fixAssignments -fixMeta -fixHdfsHoles

• Assignee: Umesh Agashe
• Reporter: Michael Stack
• Created: 12/Sep/17 04:28
• Updated: 22/Oct/18 17:36
• Resolved: 18/Apr/18 22:23

Description

hbck needs updating to run against hbase2. Meantime, if an hbck from hbase1 is run against hbck2, it may do damage. hbase2 should defend itself against hbck1 ops.

Issue Links

• links to: reviewboard