[HBASE-19925] Delete an unreachable peer will triggers all regionservers abort

• Type: Bug
• Status: Patch Available
• Priority: Critical
• Resolution: Unresolved
• Assignee: Yun Zhao
• Reporter: Yun Zhao
• Created: 03/Feb/18 03:33
• Updated: 25/Jun/18 02:50

Description

Add an unreachable peer

add_peer '4', CLUSTER_KEY => "server1.cie.com:2181:/hbase"

After a while to delete it,Regionserver will appear in the following log and stop.

2018-02-02 20:04:25,959 INFO [main-EventThread.replicationSource,4] regionserver.ReplicationSource: Replicating 5467de52-dc46-45be-902c-110dd7a83e06 -> null
2018-02-02 20:04:25,960 ERROR [main-EventThread.replicationSource,4.replicationSource.xxxx.com%2C16020%2C1515498473547.default,4] regionserver.ReplicationSource: Unexpected exception in ReplicationSourceWorkerThread, currentPath=null
java.lang.IllegalArgumentException: Peer with id= 4 is not connected
 at org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.getStatusOfPeer(ReplicationPeersZKImpl.java:207)
 at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.isPeerEnabled(ReplicationSource.java:327)
 at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource$ReplicationSourceWorkerThread.run(ReplicationSource.java:512)
2018-02-02 20:04:25,960 INFO [main-EventThread.replicationSource,4.replicationSource.xxxx.com%2C16020%2C1515498473547.default,4] regionserver.HRegionServer: STOPPED: Unexpected exception in ReplicationSourceWorkerThread

 

HBase 1.2.6

Issue Links

• relates to: HBASE-17381 ReplicationSourceWorkerThread can die due to unhandled exceptions