Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
2011.01
-
None
-
Fuse ESB 4.3.03
Description
We have discovered a memory leak with the CXFSE proxy bean, which is causing
load tests for our Servicemix application to fail over time.
The issue seems to be related to how CXF org.apache.cxf.endpoint.ClientImpl
manages something called the "response context". This class copies all
property values of the caller's message to the response context, including
the marshalled response. In the case of Servicemix CXFSE, this means that
the actual JBI MessageExchange is copied to the response context.
The leak occurs because the client maintains a thread-based map of thread
to Map<>, which is provided as the response context should the caller
choose to ask for it. In the case of the CXFSE proxy bean, most use
cases would never ask, since they are programmatically wired into the
application. Granted, each invocation of a method on the proxy can
clear the prior response context, but in our case with many threads
calling methods across a dozen proxies, soon each client is holding
onto a message exchange, including the sent and received payload over
the NMR.
In our application, we actually create a CxfSeProxyFactoryBean manually
to create proxies to some of our services within our testing framework.
For these proxies, I was able to remove some of this leak by adding an
interceptor to the CXF interceptor chain, very late in the list of
interceptors, to clear the response context:
public static class ContextCleanupInterceptor extends AbstractPhaseInterceptor<Message>
{
private Client client;
public ContextCleanupInterceptor( Client client )
{ super( Phase.POST_LOGICAL_ENDING ); this.client = client; } @Override
public void handleMessage( Message message ) throws Fault
@Override
public void handleFault( Message message )
{ client.getResponseContext().clear(); }
}
And add this to the proxy. Unfortunately, there is no way to install this interceptor
decleratively that I know of, since the "client" in this case is constructed by the
bean and not exposed to spring. Many of our services have <cxfse:proxy> spring bean
wirings that we cannot intercept in this way.
Since this is such a major potential problem, I would suggest adding a similar interceptor
to the client "port" proxy within CxfSeProxyFactoryBean such that the response context
is cleaned up this way. At the very least, any property values of the response context
such as "org.apache.servicemix.*" should be removed. I have noticed that both the
request and response context at times also receive a property called "java.lang.ref.Method",
holding onto the method being called. This is really bad for the perm gen on redeploy
of an application, since this can cause prior classes to be leaked on the permanent generation
in the JVM.