[HBASE-26219] Negative time is logged while waiting on regionservers

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Fix Version/s: 2.5.0, 3.0.0-alpha-2, 2.4.6, 2.3.7
• Labels: beginner
• Assignee: Levente Palasthy
• Reporter: Peter Somogyi
• Created: 24/Aug/21 10:44
• Updated: 26/Aug/21 18:49
• Resolved: 25/Aug/21 14:59

Description

On Master startup when waiting on RegionServers to report in the log message displays a negative time for the lastChange time.
 

2021-07-30 06:49:02,882 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=1502ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-1502ms
2021-07-30 06:49:03,693 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:04,384 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=3004ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-3004ms
2021-07-30 06:49:04,694 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:05,694 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:05,886 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=4506ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-4506ms
2021-07-30 06:49:06,695 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:07,389 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=6009ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-6009ms
2021-07-30 06:49:07,695 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:08,700 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:08,891 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=7511ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-7511ms
2021-07-30 06:49:09,701 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:10,393 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=9013ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-9013ms
2021-07-30 06:49:10,701 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:11,702 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:11,896 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=10516ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-10516ms
2021-07-30 06:49:12,702 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:13,398 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=12018ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-12018ms
2021-07-30 06:49:13,703 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:14,703 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:14,900 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=13520ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-13520ms
2021-07-30 06:49:15,704 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:16,402 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=0; waited=15022ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=-15022ms
2021-07-30 06:49:16,704 WARN org.apache.hadoop.hbase.master.assignment.AssignmentManager: No servers available; cannot place 1 unassigned regions.
2021-07-30 06:49:16,778 INFO org.apache.hadoop.hbase.master.ServerManager: Registering regionserver=example.com,16020,1627627573364
2021-07-30 06:49:16,803 INFO org.apache.hadoop.hbase.master.ServerManager: Waiting on regionserver count=1; waited=15423ms, expecting min=1 server(s), max=NO_LIMIT server(s), timeout=4500ms, lastChange=0ms 

Issue Links

• links to: GitHub Pull Request #3622