Bug 42366 - Memory leak in newer mod_jk version when connection is abortet by the client
Summary: Memory leak in newer mod_jk version when connection is abortet by the client
Status: RESOLVED INVALID
Alias: None
Product: Tomcat Connectors
Classification: Unclassified
Component: Common (show other bugs)
Version: unspecified
Hardware: PC All
: P2 normal (vote)
Target Milestone: ---
Assignee: Tomcat Developers Mailing List
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-05-09 07:04 UTC by Tom Lamal
Modified: 2011-10-25 18:22 UTC (History)
0 users



Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Tom Lamal 2007-05-09 07:04:18 UTC
I have a web application with a chat subsystem. Chat is implemented by 
streaming a never ending page to the client, with new chat messages 
subsequently added. The server recognizes if a user has left by a write to the 
outstream resulting in an exception. The server then closes the connection and 
the serving thread is returned to the thread pool.

While all standard (normally terminating) pages of the web application work 
fine, the chat page, while working as expected, results in the apache process 
bloating up, up to a gigabyte.

I have ruled out hung tomcat threads, since several hundreds of thousands of 
requests can be served, until memory consumption makes the system fail.

The problem occurs on FreeBSD and Linux with Apache 2.0.55 worker_mpm and 
mod_jk versions 1.2.14, 1.2.19 and 1.2.22.

The problem does not seem to exist in mod_jk 1.2.6. Using this version, the 
Apache process stays small (less than 30 MB).
Comment 1 Rainer Jung 2007-05-09 11:35:03 UTC
I tried to reproduce, but did not succeed. So please give a couple of info:

- mod_jk config (workers.properties, Jk* directives)

During your test: 

- does the number of apache processes go up, or is it stable?
- does the number of backend connections go up, or is it stable?

As I understand your report, neither Apache threads, nor backend connections get
hung, but the apache processes grow in size?

Any chance, you retry with Apache 2.0.59?

Furthermore: Could you please save /proc/PID/maps before, after and a few times
during the test, where PID is the PID of one of the Apache children, that
handles the requests (which one gets the requests could be either logged with %P
in the access log, or you simply look at mod_jk log lines, which contain the pid
as a field. Those files are ASCII and can be attached to this BZ.

For my tests, I used a simple test jsp, of the form:

<%@page session="false"%>
<%
long sleepDuration=0;
String sleepDurationParameter = request.getParameter("dur");
if ( sleepDurationParameter != null && ! sleepDurationParameter.equals("") ) {
        sleepDuration=Long.valueOf(sleepDurationParameter).longValue();
}
%>
Parameter: <%= sleepDuration %>
Begin Sleeping ...
<%
Thread.sleep(sleepDuration);
%>
Begin Payload
... (about 20K nonsense characters) ...
The End

Could you retry with that? I called it like .../sl.jsp?dur=3000 with a client,
which had a wait timeout of 1 second, I can force connection aborts and
respective error messages, but get not mem leak.

What info/warn/error/emerg messages does your jk log file show?

 
Comment 2 Rainer Jung 2008-01-01 21:40:11 UTC
Could you please check, if the problem is fixed in the new version 1.2.26?
Comment 3 Tom Lamal 2008-04-05 11:23:45 UTC
I just tried it out with the current versions of mod_jk (1.2.26) and tomcat (5.5.26).

The problem is still the same.

Top reports each of the apache processes using more than 150 MB of "RES" memory (and increasing), while with mod_jk 1.2.6 they each use around 10 MB (staying constant).

Additionally I noticed, that the problem only seems to appear with the following option activated:

JkOptions     +FlushPackets
Comment 4 Tom Lamal 2008-04-05 11:25:06 UTC
And I'm sorry. It is still Apache 2.0.55 
Comment 5 Tom Lamal 2008-04-05 11:41:15 UTC
Please excuse the many posts.

> - mod_jk config (workers.properties, Jk* directives)

JK* directives:

   JkWorkersFile /etc/apache2/workers.properties
   JkLogFile  /var/log/jk.log
   JkLogLevel error
   JkOptions     +FlushPackets

(as noted above, it seems to work without +FlushPackets)

workers.properties:
worker.list=worker1, worker2

worker.worker2.port=8010
worker.worker2.host=127.0.0.1
worker.worker2.type=ajp13

worker.worker1.port=8009
worker.worker1.host=127.0.0.1
worker.worker1.type=ajp13

(worker2 is not in use, there is not even a tomcat responding on that port)


> - does the number of apache processes go up, or is it stable?
it is stable, and the number is the same with both the working and the problematic version of mod_jk

> - does the number of backend connections go up, or is it stable?
I think it is stable.

the number of client connections, measured with
netstat | grep www | grep ESTA | wc -l
is the same also.

> Any chance, you retry with Apache 2.0.59?
I will try to do that.

> What info/warn/error/emerg messages does your jk log file show?
whenever a connection is dropped by the client, the following error message appears in the jk.log file (it is the same for version 1.2.6 and 1.2.26:

[jk_ajp_common.c (1462)]: ERROR: Client connection aborted or network problems
Comment 6 Rainer Jung 2008-04-07 06:55:39 UTC
I couldn't reproduce the leak, even with FlushPackets.

Can you provide a simple webapp to reproduce it?
Comment 7 Rainer Jung 2011-10-25 18:22:20 UTC
No response for 3.5 years.
Closing as invalid.