[XMLRPC-166] Misbehaving clients can hang ServletWebServer

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.1
• Fix Version/s: 3.1.2
• Assignee: Jochen Wiedmann
• Reporter: Alan Burlison
• Created: 08/Apr/09 22:17
• Updated: 29/Apr/09 20:03
• Resolved: 09/Apr/09 07:22

Description

We are using the ServletWebServer class, and we are getting occasional hangs, where the listener thread gets stuck as follows:

"XML-RPC Weblistener" prio=3 tid=0x089bd000 nid=0x15 runnable [0xe22ce000..0xe22ceb60]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:293)
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:331)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:789)

• locked <0xeb722510> (a java.lang.Object)
  at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:746)
  at com.sun.net.ssl.internal.ssl.AppInputStream.read(AppInputStream.java:75)
• locked <0xeb722838> (a com.sun.net.ssl.internal.ssl.AppInputStream)
  at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
  at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
• locked <0xeb72ae70> (a org.apache.xmlrpc.webserver.HttpServletRequestImpl$1)
  at org.apache.xmlrpc.webserver.HttpServletRequestImpl$2.read(HttpServletRequestImpl.java:94)
  at javax.servlet.ServletInputStream.readLine(ServletInputStream.java:94)
  at org.apache.xmlrpc.webserver.HttpServletRequestImpl.readLine(HttpServletRequestImpl.java:170)
  at org.apache.xmlrpc.webserver.HttpServletRequestImpl. (HttpServletRequestImpl.java:106)
  at org.apache.xmlrpc.webserver.ServletConnection. (ServletConnection.java:50)
  at org.apache.xmlrpc.webserver.ServletWebServer.newTask(ServletWebServer.java:145)
  at org.apache.xmlrpc.webserver.WebServer.run(WebServer.java:335)
  at java.lang.Thread.run(Thread.java:619)

The problem is this: when an incoming connection comes in, a new HttpServletRequestImpl object is created. The constructor of that class reads in the HTTP headers. The problem with that is that the HttpServletRequestImpl constructor runs in the same thread that accepts incoming connections, so if it blocks for any reason (e.g. the client hangs) it blocks the accept thread, which means that any new connections also hang.

I'm therefore proposing that the reading of the HTTP headers be factored out of the HttpServletRequestImpl constructor and into a new method that is called from the 'run' method of the thread that handles the request. That way, a client that hangs will only hang the thread that is servicing it and not the entire XML-RPC server.

Can anyone think of any reasons why this might cause problems? If not I'll do the changes and submit a patch.

I also note that Connection class does this:

// set read timeout to 30 seconds
socket.setSoTimeout (30000);

That should mean that the maximum time a client can hang the server for is 30 seconds, but by observation that's clearly not happening and needs further investigation.