[HBASE-21164] reportForDuty to spew less log if master is initializing

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 1.5.0, 2.2.0, 2.1.1
• Component/s: regionserver
• Assignee: Mingliang Liu
• Reporter: Michael Stack
• Created: 06/Sep/18 21:04
• Updated: 21/Feb/19 10:34
• Resolved: 25/Sep/18 18:33

Description

RegionServers do reportForDuty on startup to tell Master they are available. If Master is initializing, and especially on a big cluster when it can take a while particularly if something is amiss, the log every three seconds is annoying and doesn't do anything of use. We should spew less those logs. Here is example:

2018-09-06 14:01:39,312 INFO org.apache.hadoop.hbase.regionserver.HRegionServer: reportForDuty to master=vc0207.halxg.cloudera.com,22001,1536266763109 with port=22001, startcode=1536266763109
2018-09-06 14:01:39,312 WARN org.apache.hadoop.hbase.regionserver.HRegionServer: reportForDuty failed; sleeping and then retrying.
....

For example, I am looking at a large cluster now that had a backlog of procedure WALs. It is taking a couple of hours recreating the procedure-state because there are millions of procedures outstanding. Meantime, the Master log is just full of the above message – every three seconds...

Issue Links

• breaks: HBASE-21940 TestMasterShutdown fails on branch-1
• links to: Review Board (branch-2.1)