[HBASE-20867] RS may get killed while master restarts

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1, 2.1.0, 2.0.1
• Fix Version/s: 3.0.0-alpha-1, 2.2.0, 2.1.1, 2.0.2
• Assignee: Allan Yang
• Reporter: Allan Yang
• Created: 10/Jul/18 14:53
• Updated: 04/Mar/19 09:08
• Resolved: 25/Jul/18 10:17

Description

If the master is dispatching a RPC call to RS when aborting. A connection exception may be thrown by the RPC layer(A IOException with "Connection closed" message in this case). The RSProcedureDispatcher will regard is as an un-retryable exception and pass it to UnassignProcedue.remoteCallFailed, which will expire the RS.
Actually, the RS is very healthy, only the master is restarting.
I think we should deal with those kinds of connection exceptions in RSProcedureDispatcher and retry the rpc call