On a 3-node cluster with RF=2, decommissioning a node may cause quorum write timeout because messaging backlog to decommissioned node is cleared via Gossiper#removeEndpoint() -> OutboundTcpConnection#closeSocket().
(Timeout is less likely to happen with RF=3, because we can afford one less response)
We can avoid it by delaying to destroy messaging connection so that messages are sent and responded. This patch also avoids reopening already closed connection on MessagingService#convict().
New messaging framework rewrite in Trunk avoids the issues by not clearing messaging backlog.