ODE
  1. ODE
  2. ODE-836

OutOfMemoryError has occured if many processes are deployed.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.3.3
    • Fix Version/s: None
    • Component/s: BPEL Runtime
    • Labels:
    • Environment:
      ODE-1.3.3+Tomcat

      Description

      Though dehydration is enabled,and there are no process-instances,
      if many processes are deployed, OutOfMemoryError has occured.

      You can see the mail:
      http://mail-archives.apache.org/mod_mbox/ode-user/201005.mbox/%3CAANLkTikaw5Rmx99ui3bKj6leheKO96yLwW6AtdIKjL2W@mail.gmail.com%3E

      1. ThreadDump(314).txt
        24 kB
        Rick.Todo
      2. minimize.zip
        2 kB
        Rick.Todo

        Activity

        Hide
        Veresh Jain added a comment -

        Do you have thread dump ?
        512m for deploying so many process is not recommended.

        Show
        Veresh Jain added a comment - Do you have thread dump ? 512m for deploying so many process is not recommended.
        Hide
        Rick.Todo added a comment -

        Thanks,Veresh.
        Please see attached threaddump.
        It shows threaddump when 314 processes were deployed to tomcat+ode-1.3.3(-Xmx512m) with enabled dehydration.
        and there were no process-instance.
        ode-axis2.process.dehydration=true
        ode-axis2.process.dehydration.maximum.age=600000000
        ode-axis2.process.dehydration.maximum.count=50
        And then, OutOfMemoryError has occured with 324 processes.

        We want to deploy whole 1000 processes.
        How much heap-memory ode need to be deployed 1000 process?, 2GB or more?

        Show
        Rick.Todo added a comment - Thanks,Veresh. Please see attached threaddump. It shows threaddump when 314 processes were deployed to tomcat+ode-1.3.3(-Xmx512m) with enabled dehydration. and there were no process-instance. ode-axis2.process.dehydration=true ode-axis2.process.dehydration.maximum.age=600000000 ode-axis2.process.dehydration.maximum.count=50 And then, OutOfMemoryError has occured with 324 processes. We want to deploy whole 1000 processes. How much heap-memory ode need to be deployed 1000 process?, 2GB or more?
        Hide
        Rafal Rusin added a comment -

        How many of those 1000 processes have active instances?

        Show
        Rafal Rusin added a comment - How many of those 1000 processes have active instances?
        Hide
        Veresh Jain added a comment -

        2 GB should be enough.

        Show
        Veresh Jain added a comment - 2 GB should be enough.
        Hide
        Julia Chang added a comment -

        default value of "process.hydration.lazy" is set as "true", but
        default return value of org.apache.ode.bpel.engine.BpelServerImpl#isLazyHydratable is "false".
        So, all processes are deployed as hydrated.

        Inequality sign of this is NOT correct?
        >return process.getEstimatedHydratedSize() < _hydrationLazyMinimumSize;

        Show
        Julia Chang added a comment - default value of "process.hydration.lazy" is set as "true", but default return value of org.apache.ode.bpel.engine.BpelServerImpl#isLazyHydratable is "false". So, all processes are deployed as hydrated. Inequality sign of this is NOT correct? >return process.getEstimatedHydratedSize() < _hydrationLazyMinimumSize;
        Hide
        Julia Chang added a comment -

        1.org.apache.axis2.description.AxisService#releaseSchemaList() is called when a process is undeployed.
        If many processes are deployed, much heap memory was needed.
        Should not releaseSchemaList() be called after BindingContextImpl#createService()?

        2.org.apache.axis2.wsdl.util.WSDLWrapperBasicImpl is used as a default implementation of WSDLWrapperImpl.
        I think WSDLWrapperReloadImpl is better if there are many processes,
        but I can't configure axis2.xml(reduceWSDLMemoryCache,reduceWSDLMemoryType).
        Please teach me to configure axis2 to use WSDLWrapperReloadImpl.

        3. org.apache.ode.store.DeploymentUnitDir#getDocRegistry() loads WSDL definitions,
        and keeps them in heap memory with each process.
        _docRegistry should be set "null" when the process is dehydrated.

        Show
        Julia Chang added a comment - 1.org.apache.axis2.description.AxisService#releaseSchemaList() is called when a process is undeployed. If many processes are deployed, much heap memory was needed. Should not releaseSchemaList() be called after BindingContextImpl#createService()? 2.org.apache.axis2.wsdl.util.WSDLWrapperBasicImpl is used as a default implementation of WSDLWrapperImpl. I think WSDLWrapperReloadImpl is better if there are many processes, but I can't configure axis2.xml(reduceWSDLMemoryCache,reduceWSDLMemoryType). Please teach me to configure axis2 to use WSDLWrapperReloadImpl. 3. org.apache.ode.store.DeploymentUnitDir#getDocRegistry() loads WSDL definitions, and keeps them in heap memory with each process. _docRegistry should be set "null" when the process is dehydrated.
        Hide
        Rick.Todo added a comment -

        attached patch can save memory usage of OProcess and its children.
        it minimize OProcess to share emptyMap,emptySet, and String.

        but, attached patch requires same memory usage as before during compiling,
        it is better to share shareable objects during compiling.

        Show
        Rick.Todo added a comment - attached patch can save memory usage of OProcess and its children. it minimize OProcess to share emptyMap,emptySet, and String. but, attached patch requires same memory usage as before during compiling, it is better to share shareable objects during compiling.
        Hide
        Tammo van Lessen added a comment -

        Hi Rick,

        thanks for the patch! Is the attached version the correct one? I'm asking because it just minimizes the footprint of OXPath10Expression although in your comment you're talking about OProcess...

        Show
        Tammo van Lessen added a comment - Hi Rick, thanks for the patch! Is the attached version the correct one? I'm asking because it just minimizes the footprint of OXPath10Expression although in your comment you're talking about OProcess...
        Hide
        Tammo van Lessen added a comment -

        Also, wouldn't calling .intern() on a String do the same?

        Show
        Tammo van Lessen added a comment - Also, wouldn't calling .intern() on a String do the same?
        Hide
        Rick.Todo added a comment -

        Hi, Tammo.
        it's for ODE-1.3.3, not for trunk, sorry.
        it works on ODE-1.3.3 but I did not applied it on any other version.

        for trunk, it is better to share shareable objects during compiling and loadinng cbp.
        but ode's compile process is complex, i have not did it.
        i hope someone improve it.

        Show
        Rick.Todo added a comment - Hi, Tammo. it's for ODE-1 .3.3, not for trunk, sorry. it works on ODE-1 .3.3 but I did not applied it on any other version. for trunk, it is better to share shareable objects during compiling and loadinng cbp. but ode's compile process is complex, i have not did it. i hope someone improve it.

          People

          • Assignee:
            Unassigned
            Reporter:
            Julia Chang
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development