[ZOOKEEPER-562] c client can flood server with pings if tcp send queue filled

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.2.1
• Fix Version/s: 3.1.2, 3.2.2, 3.3.0
• Component/s: c client
• Hadoop Flags: Reviewed

Release Note

Basically the problem here is that the client gets confused, it tries to send a ping to the server but since the tcp queue is full it's unable to do so. The logic responsible for sending occasional pings based on the timeout gets confused by this, and ends up flooding the server with pings. Eventually this clears up, however it can result in increased load on the server and instability for the effected client.

• Assignee: Benjamin Reed Assign to me
• Reporter: Patrick D. Hunt
• Created: 26/Oct/09 20:45
• Updated: 26/Mar/10 17:25
• Resolved: 29/Oct/09 21:35

Description

The c client can flood the server with pings if the tcp queue is filled.

Say the cluster is overloaded and shuts down the recv processing

a c client can send a ping, but since last_send is only updated on successful pushing of data into the
socket, if flush_send_queue fails to send any data (send_buffer returns 0) then last_send is not updated
and zookeeper_interest will again send a ping the next time it is woken - which could be 0 if recv_to is close
to 0, easily could happen if server is not sending data to the client.