mod_python
  1. mod_python
  2. MODPYTHON-235

Memory leaks in main Apache process when doing 'restart' or 'graceful'.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 3.3.1
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None

      Description

      As raised in thread:

      http://www.modpython.org/pipermail/mod_python/2007-June/023926.html

      it looks like mod_python exhibits memory leaks in the main Apache process when a 'restart' or 'graceful' is done. Ie., when only child process are killed off and restarted rather than whole Apache being stopped.

      The poster though seems to be of opinion that it only happens when certain auth modules are loaded, which makes the behaviour somewhat odd if true. One would expect it simply to always leak rather than being dependent on the other modules existing.

      As it stands, the whole mod_python initialisation really needs to be done over because of issues such as MODPYTHON-195. If done it may resolve this problem.

        Activity

        Hide
        Graham Dumpleton added a comment -

        These memory leaks are due to mod_python not calling Py_Finalize() call on active Python interpreter instance. Result being that memory isn't released. After restart Py_Initialize() is called again with memory being allocated again. Thus see an incremental memory leak on every restart.

        Code needs to register a cleanup callback on appropriate memory pool and destroy interpreter properly prior to internal restart.

        Show
        Graham Dumpleton added a comment - These memory leaks are due to mod_python not calling Py_Finalize() call on active Python interpreter instance. Result being that memory isn't released. After restart Py_Initialize() is called again with memory being allocated again. Thus see an incremental memory leak on every restart. Code needs to register a cleanup callback on appropriate memory pool and destroy interpreter properly prior to internal restart.
        Hide
        Michael P. Soulier added a comment -

        Is anyone planning to fix this issue? It's hitting a product that I work on that performs graceful restarts on apache regularly.

        If the solution above works, would the fix not be simple? This issue is 2 years old.

        Show
        Michael P. Soulier added a comment - Is anyone planning to fix this issue? It's hitting a product that I work on that performs graceful restarts on apache regularly. If the solution above works, would the fix not be simple? This issue is 2 years old.
        Hide
        Graham Dumpleton added a comment -

        No plans to fix it. No plans for any work to be done on mod_python by anyone.

        The fix as described also isn't sufficient anyway as Python itself leaks memory even when Py_Initialize() is called. For mod_python 3.X that is by design. For Python 2.X, which is all mod_python supports anyway, not clear if the memory leaks in Python are by design or unintended. Could perhaps be a mix of both.

        If you are using mod_python purely for purposes of hosting a Python web application which also has a WSGI interface, you should consider moving to mod_wsgi. The mod_wsgi package ensure that Py_Finalize() is called and so that aspect of leak doesn't exist. There is still the issue with Python itself leaking, but in mod_wsgi 3.X that is avoided by delaying initialisation of Python until after processes fork.

        Do note that although you can use mod_python and mod_wsgi in process together, you will still be subject to memory leaks because mod_python behaviour takes precedence and so it will still leak. Thus, only use mod_wsgi alone and don't use mod_python at the same time.

        Show
        Graham Dumpleton added a comment - No plans to fix it. No plans for any work to be done on mod_python by anyone. The fix as described also isn't sufficient anyway as Python itself leaks memory even when Py_Initialize() is called. For mod_python 3.X that is by design. For Python 2.X, which is all mod_python supports anyway, not clear if the memory leaks in Python are by design or unintended. Could perhaps be a mix of both. If you are using mod_python purely for purposes of hosting a Python web application which also has a WSGI interface, you should consider moving to mod_wsgi. The mod_wsgi package ensure that Py_Finalize() is called and so that aspect of leak doesn't exist. There is still the issue with Python itself leaking, but in mod_wsgi 3.X that is avoided by delaying initialisation of Python until after processes fork. Do note that although you can use mod_python and mod_wsgi in process together, you will still be subject to memory leaks because mod_python behaviour takes precedence and so it will still leak. Thus, only use mod_wsgi alone and don't use mod_python at the same time.
        Hide
        Michael P. Soulier added a comment -

        Understood.

        It might help to update http://www.modpython.org/ with this information.

        Thank you,
        Mike

        Show
        Michael P. Soulier added a comment - Understood. It might help to update http://www.modpython.org/ with this information. Thank you, Mike
        Hide
        Graham Dumpleton added a comment -

        Have no access to www.modpython.org to make changes.

        Show
        Graham Dumpleton added a comment - Have no access to www.modpython.org to make changes.

          People

          • Assignee:
            Unassigned
            Reporter:
            Graham Dumpleton
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development