[MODPYTHON-1] Circular references starting from the request object create a memory leak

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.10, 3.1.3
• Fix Version/s: 3.2.7
• Assignee: Nicolas Lehuen
• Reporter: Nicolas Lehuen
• Created: 14/Dec/04 06:51
• Updated: 05/Mar/06 13:32
• Resolved: 19/Dec/04 19:41

Description

Hi,

I'm running Apache 2.0.50, mod_python 3.1.3 with Python 2.3.4 on Windows XP Pro and Windows 2000 server.

I finally found a source for the memory leaks I have been observing for a while now : if you store an object in the request object that reference directly or indirectly the request object itself, it builds a circular reference. This circular reference does not seem to be garbage collected.

The test handler :

from mod_python import apache

def handler(req):

1. req.foobar=req
   req.content_type='text/html'
   req.write('OK')
   return apache.OK

I used Apache Benchmark :

ab -c 5 -n 100000 http://localhost/test.py

With the comment line, everything is OK, I do not see any memory leak.

If I uncomment the 'req.foobar=req' line, the memory leak shows up. My Apache process normally uses around 20 MB without the comment, and in 30 seconds of running the above handler (comment removed) with the above Apache Benchmark command, it reaches 125 MB !

Normally, circular reference are collected by the garbage collector of Python, but unfortunately when defining object from C, you have to do a few tricks to support this :

http://www.python.org/doc/2.3.4/api/supporting-cycle-detection.html

It seems that this is not done in mod_python, by having a look at requestobject.c.

I find this quite a big problem, and I don't find any workaround except "Don't reference the request object from anywhere that might be referenced by the request object", which is a pretty big limitation, quite annoying in the application I'm building.