Cayenne
  1. Cayenne
  2. CAY-1668

Memory Exhaustion Problem with Deserialization of ObjectContext

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.2
    • Fix Version/s: 3.1B1
    • Component/s: Core Library
    • Labels:
      None
    • Environment:
      MacOS-X 10.7, Java 1.6.0_29, Simple Web Application run in the Maven plugin "jetty-maven-plugin"

      Description

      I am running a simple web application launched from maven using "jetty-maven-plugin". The simple web application has a Filter which does this;

      1) Copies the HttpSession's attributes to a Map
      2) Serializes the Map
      3) Deserializes the Map
      4) Copies the entries of the Map back into HttpSession

      In each case, the ClassLoader should be the same (no stranded singletons etc...) and there is no "hot deploy" happening. The purpose of this undertaking is to ensure that the application is able to handle serialized sessions. In doing this experiment, I have observed a memory exhaustion issue around serializing and de-serializing ObjectContext. If I explicitly stop serializing "ObjectContext" then the problem stops.

      Using jprofiler, I have observed that the memory consumption increases roughly exponentially in relation to the number of serialization/de-serialization events. Also from jquery analysis I see the memory is being referenced from;

      org.apache.commons.collections.map.AbstractHashedMap$HashEntry[]
      > org.apache.commons.collecitons.map.LRUMap
      >> org.apache.cayenne.access.jdbc.SQLTemplateResourceManager
      >>> ...velocity

      The problem manifests itself quite quickly; a few dozen serialize + de-serialize phases.

        Activity

        Hide
        Andrew Lindesay added a comment -

        It does appear that the problem is in an Apache Commons abstract superclass. What is happening is that whilst running;

        AbstractReferenceMap.doReadObject(..)

        ...the method...

        AbstractReferenceMap.checkCapacity(..)

        ...is invoked and it seems to be doubling the size of the Map's entry array on de-serialization. Through sequential serialization + de-serialization this yields the exponential growth in the memory consumption.

        It seems that the problem is caused by the 'threshold' value being calculated after to populating the Map during de-serialization.

        I have re-built 3.0-STABLE with commons 3.2.1 and it still seems to exhibit the same problem.

        The release cycles for commons seems to be quite long these days. Hmmm.....

        Show
        Andrew Lindesay added a comment - It does appear that the problem is in an Apache Commons abstract superclass. What is happening is that whilst running; AbstractReferenceMap.doReadObject(..) ...the method... AbstractReferenceMap.checkCapacity(..) ...is invoked and it seems to be doubling the size of the Map's entry array on de-serialization. Through sequential serialization + de-serialization this yields the exponential growth in the memory consumption. It seems that the problem is caused by the 'threshold' value being calculated after to populating the Map during de-serialization. I have re-built 3.0-STABLE with commons 3.2.1 and it still seems to exhibit the same problem. The release cycles for commons seems to be quite long these days. Hmmm.....
        Hide
        Andrus Adamchik added a comment -

        I think we can improve deserialization performance in general and solve this specific issue if we make ObjectStore.objectMap (this is the AbstractReferenceMap causing trouble) transient. There's no value in saving and restoring this map. There's another map called "changes", which is a regular HashMap, and that needs to be preserved.

        Now the implementation can be a little tricky, as "objectMap" is provided by the DI factory and we need to create an empty map of the same type on deserialization. This may require refactoring of ObjectStore creation flow. Should be doable though...

        Show
        Andrus Adamchik added a comment - I think we can improve deserialization performance in general and solve this specific issue if we make ObjectStore.objectMap (this is the AbstractReferenceMap causing trouble) transient. There's no value in saving and restoring this map. There's another map called "changes", which is a regular HashMap, and that needs to be preserved. Now the implementation can be a little tricky, as "objectMap" is provided by the DI factory and we need to create an empty map of the same type on deserialization. This may require refactoring of ObjectStore creation flow. Should be doable though...
        Hide
        Andrus Adamchik added a comment -

        BTW, after researching that I'll probably follow up with commons project and open a Jira there... Ideally they should fix it themselves at some point...

        Show
        Andrus Adamchik added a comment - BTW, after researching that I'll probably follow up with commons project and open a Jira there... Ideally they should fix it themselves at some point...
        Hide
        Andrew Lindesay added a comment - - edited

        For what it is worth; I have tested commons 3.1 and 3.2 with a small test rig and the problem appears to be gone in 3.2. By code introspection, I can't easily see how they resolved it though.

        Show
        Andrew Lindesay added a comment - - edited For what it is worth; I have tested commons 3.1 and 3.2 with a small test rig and the problem appears to be gone in 3.2. By code introspection, I can't easily see how they resolved it though.
        Hide
        Andrus Adamchik added a comment -

        This is good news. Then we can simply flip the dependency version in Cayenne POM

        Show
        Andrus Adamchik added a comment - This is good news. Then we can simply flip the dependency version in Cayenne POM
        Hide
        Andrus Adamchik added a comment -

        Should be fixed per CAY-1684

        Andrew, fill free to reopen if you keep encountering this leak.

        Show
        Andrus Adamchik added a comment - Should be fixed per CAY-1684 Andrew, fill free to reopen if you keep encountering this leak.

          People

          • Assignee:
            Unassigned
            Reporter:
            Andrew Lindesay
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development