[HBASE-26885] The TRSP should not go on when it get a bogus server name from AM

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.5.0, 3.0.0-alpha-3, 2.4.12
• Component/s: proc-v2
• Hadoop Flags: Reviewed
• Assignee: Zheng Wang
• Reporter: Zheng Wang
• Created: 25/Mar/22 11:23
• Updated: 01/Sep/22 10:57
• Resolved: 04/Apr/22 12:32

Description

Currently it will submit lots of unnecessary OpenRegionProcedure by retry.

Related log looks like below, 'localhost,1,1' is the bogus server:

2022-03-22 10:17:48,301 WARN  [PEWorker-8] assignment.RegionRemoteProcedureBase: Can not add remote operation pid=17952, ppid=17951, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure for region {ENCODED => 490391c232c7aa13f7e0d50bfe1f7235, NAME => 'TestTable1,0000000000000000002497747,1647568640784.490391c232c7aa13f7e0d50bfe1f7235.', STARTKEY => '0000000000000000002497747', ENDKEY => ''} to server localhost,1,1, this usually because the server is alread dead, give up and mark the procedure as complete, the parent procedure will take care of this.
org.apache.hadoop.hbase.procedure2.NoServerDispatchException: localhost,1,1; pid=17952, ppid=17951, state=RUNNABLE, locked=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure
        at org.apache.hadoop.hbase.procedure2.RemoteProcedureDispatcher.addOperationToNode(RemoteProcedureDispatcher.java:168)
        at org.apache.hadoop.hbase.master.assignment.RegionRemoteProcedureBase.execute(RegionRemoteProcedureBase.java:285)
        at org.apache.hadoop.hbase.master.assignment.RegionRemoteProcedureBase.execute(RegionRemoteProcedureBase.java:58)
        at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:962)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1648)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1395)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:78)
        at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1965)
2022-03-22 10:17:48,301 DEBUG [PEWorker-8] procedure2.RootProcedureState: Add procedure pid=17952, ppid=17951, state=SUCCESS, locked=true; org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure as the 8th rollback step 

Issue Links

• links to: GitHub Pull Request #4276
• links to: GitHub Pull Request #4299