ODE
  1. ODE
  2. ODE-943

NoClassDefFoundError for org.apache.log4j.helpers.AbsoluteTimeDateFormat in SimpleScheduler.doLoadImmediate() leads memory leak

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.5, 1.4
    • Fix Version/s: 1.3.6, 1.4
    • Labels:
      None
    • Environment:
      Ubuntu 11.04 64bit, Oracle JDK 1.6, Servicemix 4.3.0

      Description

      On a fresh installation of Servicemix 4.3.0 does not expose org.apache.log4j.helpers by default although Pax logging bundle contains this package. At line:707 of Ode_1.3.5 version and line:732 of trunk, AbsoluteTimeDateFormat f = new AbsoluteTimeDateFormat(); line leads to NoClassDefFoundError for org.apache.log4j.helpers.AbsoluteTimeDateFormat. As this error is a subclass of throwable, it is not caught in catch() block.

      This issue leads to huge memory leak under load as _processedSinceLastLoadTask.clear(); line can not be reached ever, 1 million requests will make your 1gb heap size drained and get your servicemix in a fullgc cycles.

      I worked around this issue by wrapping log4j jar into servicemix by karaf@ osgi:install -s wrap:mvn:log4j/log4j/1.2.13 , and restarting whole system.

      Also log4j dependencies are imported as resolution:optional in MANIFEST-MF in ode-jbi bundle.

      To reproduce;

      1 - reduce heap size in servicemix executable
      2 - add jvm option : -verbosegc to watch gc and fullgc cycles on karaf console or -Xloggc:somefile.out for tailing a file for output
      3 - set KARAF_DEBUG=true in case you want to debug and see _processedSinceLastLoadTask.size() shows increasing number of entries
      3 - download servicemix 4.3.0 from apache
      4 - in karaf console, features:install ode
      5 - deploy a simple bpel flow, features:install examples-ode-ping-pong
      6 - generate load either via JMeter or Soapui
      7 - take a heap dump either via jmap or via JConsole finding mbean com.sun.management->HotSpotDiagnostic-> operations --> dumpHeap(heapdump.snapshot) , you can find dump file under $SMX_HOME
      8 - analyse it via yourkit profiler or jprofiler, whatever suits you, you will see one object retain most of the memory, which is org/apache/ode/scheduler/simple/SimpleScheduler on object explorer window.

        Activity

        Abdulkadir Yaman created issue -
        Tammo van Lessen made changes -
        Field Original Value New Value
        Fix Version/s 1.3.6 [ 12323509 ]
        Fix Version/s 1.4 [ 12315144 ]
        Tammo van Lessen made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Assignee Tammo van Lessen [ vanto ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Tammo van Lessen
            Reporter:
            Abdulkadir Yaman
          • Votes:
            1 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development