Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
3.4.5, 3.5.0
-
None
-
Linux, Java 1.7
Description
In our system we encountered the following problem:
If the system is stable, and there is no leader election, the leader election port connections are open for very long time without any packets being sent on them.
Some network elements silently drop the established TCP connection after a timeout if there are no packets being sent on it. In this case the ZK servers will not notice the connection loss. This causes additional delay later when the next leader election is started, as the TCP connections are not alive any more.
We would like to be able to enable TCP keepalive on the leader election sockets in order to prevent the connection timeout in some network elements due to connection inactivity.
This could be controlled by adding a new config parameter called tcpKeepAlive in the ZooKeeper configuration file. It would be only applicable in case of algorithm 3 (TCP based fast leader election), having the default value false.
If tcpKeepAlive is set to true, the TCP keepalive flag should be enabled for the leader election sockets in QuorumCnxManager.setSockOpts() by calling sock.setKeepAlive(true).
We have tested this change successfully in our environment.
Please comment whether you see any problem with this. If not, I am going to submit a patch.
I've been told that e.g. Apache ActiveMQ also has a config option for similar purpose called transport.keepalive.
