Uploaded image for project: 'Axis'
  1. Axis
  2. AXIS-1771

Excessive Memory Use During Serialization/Deserialization

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.2RC2
    • None
    • Basic Architecture
    • None
    • JDK 1.4.2, Mac OS X, Linux, Windows

    Description

      Axis uses pathological amounts of memory during the serialization/deserialization process.

      We see about a 30 to 1 ratio of memory used during (de)serialization to in-memory representation of the objects being (de)serialized. So ser/deser in axis of a 2M graph of objects uses 288M of memory! Further, the memory used seems to scale linearly with the size of the object graph being serialized.

      The memory used does seem to be released once serialization is done, so this isn't a leak.

      Using the attached example code, (based on the code used to demonstrate AXIS-1423) you can see this behavior. The test automatically runs with a max heap size of 1024M, and runs out of memory serializing a 28M object graph.

      As provided, it generates the following output:

      Buildfile: build.xml

      build:
      [javac] Compiling 1 source file to /Users/pietro/Work/Axis Memory Test/build/classes

      run:
      [java] - Unable to find required classes (javax.activation.DataHandler and javax.mail.internet.MimeMultipart). Attachment support is disabled.
      [java] Created tree with 5 levels and 3 children at each level
      [java] Axis used 13 MBytes to serialize 230 KBytes, a ratio of 30.0
      [java] GC freed 13 MBytes
      [java] Created tree with 5 levels and 4 children at each level
      [java] Axis used 71 MBytes to serialize 1 MBytes, a ratio of 31.0
      [java] GC freed 71 MBytes
      [java] Created tree with 5 levels and 5 children at each level
      [java] Axis used 288 MBytes to serialize 2 MBytes, a ratio of 51.0
      [java] GC freed 287 MBytes
      [java] Created tree with 5 levels and 6 children at each level
      [java] Axis used 671 MBytes to serialize 11 MBytes, a ratio of 29.0
      [java] GC freed 675 MBytes
      [java] Created tree with 5 levels and 7 children at each level
      [java] Out of Memory serializing 28 MBytes tree.
      [java] Java Result: 1

      BUILD SUCCESSFUL
      Total time: 2 minutes 51 seconds

      Note that the ratios are halved from the reported values, since it includes both serialization and deserialization of the object graph. So axis uses 30.5M to serialize a 1M message, and another 30.5M to deserialize it, resulting in the reported 71M reported above. Also, notice that the ratio stays close to 30 to 1 regardless of object graph size. This is the linear scaling I mention above.

      Note also, that if you tweak the code to generate very small object graphs, you see extremely high ratios, but I would expect this due to simple one-time overhead to operate on very small amounts of data. That's why I start with the graph size that I do.

      Attachments

        1. SOAPmsg_multiref_true.xml
          419 kB
          Venkat Reddy
        2. SOAPmsg_multiref_false.xml
          125 kB
          Venkat Reddy
        3. SerializationContext.java.diff
          2 kB
          Nishant Bangarwa
        4. recursion.png
          59 kB
          Venkat Reddy
        5. MTC_venkat.java
          6 kB
          Venkat Reddy
        6. memory-use-test.tgz
          1.82 MB
          Peter Molettiere
        7. memory-use-test.tgz
          2.01 MB
          Peter Molettiere
        8. MemoryTesterClient.java
          8 kB
          Peter Molettiere
        9. MemoryTesterClient.java
          11 kB
          Davanum Srinivas
        10. axis-memory-use.gif
          26 kB
          Peter Molettiere

        Activity

          People

            venkat Venkat Reddy
            petermolettiere Peter Molettiere
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: