Add Client side packet response timeout to avoid infinite wait.

I came across one issue related to Client side packet response timeout In my cluster many packet drops happened for some time.

One observation is the zookeeper client got hanged. As per the thread dump it is waiting for the response/ACK for the operation performed (synchronous API used here).
I am using zookeeper.serverCnxnFactory=org.apache.zookeeper.server.NIOServerCnxnFactory

Since only few packets missed there is no DISCONNECTED event occurred.

Need add a "response time out" for the operations or packets.

Comments from Rakesh Radhakrishnan
My observation about the problem:-

• Can use tools like 'Wireshark' to simulate the artificial packet loss.
• Assume there is only one packet in the 'outgoingQueue' and unfortunately the server response packet lost. Now, client will enter into infinite waiting. https://github.com/apache/zookeeper/blob/trunk/src/java/main/org/apache/zookeeper/ClientCnxn.java#L1515
• Probably we can discuss more about this problem and possible solutions(add packet ACK timeout or another better approach) in the jira.