[HBASE-25874] [hbase-operator-tools]Add tool for identifying "unknown servers" from master logs, then submit SCPs for each of those.

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: hbase-operator-tools-1.1.0
• Fix Version/s: hbase-operator-tools-1.2.0

Release Note

RegionsOnUnknownServersRecoverer parses the master log to identify unknown servers with regions assigned with, in meta table. This condition may happen in the event of recovering previously destroyed clusters, where new Master/RS names completely differ from the previous ones currently stored in meta table (see HBASE-24286).

NOTE: This tool is useful for clusters runing hbase versions lower than 2.2.7, 2.3.5 and 2.4.7.
For any of these versions or higher, HBCK2 'recoverUnknown' option can be used as a much simpler solution.

This tool requires the master logs path as parameter. Assuming classpath is properly set, can be run as follows:

$ hbase org.apache.hbase.RegionsOnUnknownServersRecoverer PATH_TO_MASTER_LOGS [dryRun]

The dryRun optional parameter will just parse the logs and print the list of unknown servers, without invoking hbck2 scheduleRecoveries command.

RegionsOnUnknownServersRecoverer parses master log file searching for specific messages mentioning "unknown servers". Once "unknown servers" are found, it then uses HBCK2.scheduleRecoveries to submit SCPs for each of these "unknown servers".

• Assignee: Wellington Chevreuil
• Reporter: Wellington Chevreuil
• Created: 10/May/21 10:50
• Updated: 14/May/21 18:48
• Resolved: 14/May/21 09:11

Description

Mostly useful when reusing hbase data dir from a previously destroyed cluster to a new cluster definition (see HBASE-24286) with hbase versions prior to 2.2.7, 2.3.5 and 2.4.7, where there's no support for scheduleSCPsForUnknownServers.

HBCK2 has this scheduleRecoveries method, that submits SCPs for individual RS names passed as parameter, but it still requires operators to manually parse these unknown server names from master logs in order to build the command. This tool automates the parsing step, building the list of proper server names, then reusing HBCK2.scheduleRecoveries to trigger SCPs.

Issue Links

• links to: GitHub Pull Request #86