[ZOOKEEPER-1595] Sockets should be read until exhausted

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: server
• Labels: newbie performance

Environment

Tested on Linux x64 with Oracle JDK6

• Assignee: Unassigned
• Reporter: Nikita Vetoshkin
• Created: 03/Dec/12 07:46
• Updated: 11/Dec/12 21:16

Description

doIO method in NIOServerCnxn should read (and write too) until read/write returns 0.
It's a common practice when working with non-blocking sockets. When an underlying system call (multiplexer) signals, that socket is readable, one should recv(2) all data from kernel buffer until recv fails with EAGAIN or EWOULDBLOCK.

Patch does two things (I know it's not a good idea to mix several changes, but I could stand it):

• splits doIO into doRead and doWrite
• wraps reading with while (true)

It's pretty easy to instrument the code with a counter and print how many loops we performed until the socket was not readable again.

I wrote a simple python script (http://pastebin.com/N5ifM330) which creates 6000 nodes with 5k data each, having 20 concurrent create requests in progress through one connnection.
With this script and strace attached to JVM I counted epoll_wait syscalls during the test and I got ~9500 before vs ~8000 after.
Run time measurement is very rough, but it's around ~19 secs. before vs 17.5 after.