Major Description
Hi CXF Team,
We're fighting with threading+classloading related issues when testing JBossWS-CXF integration.
The problematic part is WorkQueueManagerImpl.getAutomaticWorkQueue() method that
is reused in the following three places in CXF code base.
[/home/opalka][/home/opalka/THIRDPARTY/CXF/SVN/tags/cxf-2.2.1]>grep -r getAutomaticWorkQueue * | grep -v "\.svn" | grep -v Test | grep -v workqueue
rt/core/src/main/java/org/apache/cxf/interceptor/OneWayProcessorInterceptor.java: .getAutomaticWorkQueue().execute(new Runnable() {
rt/transports/http/src/main/java/org/apache/cxf/transport/http/HTTPConduit.java: queue = mgr.getAutomaticWorkQueue();
rt/ws/addr/src/main/java/org/apache/cxf/ws/addressing/ContextUtils.java: : workQueueManager.getAutomaticWorkQueue();
This method constructs "default" worker threads pool on first request and this thread pool is
heavily reused by CXF for other applications. We see the following problem on our server side:
When this "default" worker threads pool gets constructed all its worker threads have associated
classloader of the calling thread (in our case calling thread has associated web application classloader).
This is how Java threads are constructed (they inherit the parent thread classloader).
This thread pool is heavily reused by CXF for other applications.
The problem will appear when we undeploy the web application (the one that worker threads have associated
classloader with). After undeployment of that archive all worker threads will throw ClassNotFoundException when passed
jobs will try to do something with the classloader. Here's the sample stack trace:
2009-05-12 09:30:14,923 INFO [org.apache.cxf.phase.PhaseInterceptorChain:70] (pool-13-thread-2) Interceptor has thrown exception, unwinding now
org.apache.cxf.binding.soap.SoapFault: Problems creating SAAJ object model
at org.apache.cxf.binding.soap.saaj.SAAJInInterceptor.handleMessage(SAAJInInterceptor.java:165)
at org.apache.cxf.binding.soap.saaj.SAAJInInterceptor.handleMessage(SAAJInInterceptor.java:67)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:236)
at org.apache.cxf.endpoint.ClientImpl.onMessage(ClientImpl.java:641)
at org.apache.cxf.endpoint.ClientImpl$1$1.run(ClientImpl.java:722)
at org.apache.cxf.workqueue.SynchronousExecutor.execute(SynchronousExecutor.java:37)
at org.apache.cxf.endpoint.ClientImpl$1.onMessage(ClientImpl.java:720)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:2134)
at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream$1.run(HTTPConduit.java:2018)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: javax.xml.soap.SOAPException: Unable to create message factory for SOAP: Unable to create SAAJ meta-factoryProvider com.sun.xml.messaging.saaj.soap.SAAJMetaFactoryImpl could not be instantiated: java.lang.IllegalStateException: BaseClassLoader@298b744f
classLoader is not connected to a domain (probably undeployed?) for class com.sun.xml.messaging.saaj.soap.SAAJMetaFactoryImpl
at javax.xml.soap.MessageFactory.newInstance(Unknown Source)
at org.apache.cxf.binding.soap.saaj.SAAJInInterceptor.getFactory(SAAJInInterceptor.java:84)
at org.apache.cxf.binding.soap.saaj.SAAJInInterceptor.handleMessage(SAAJInInterceptor.java:96)
... 11 more
To prevent these kinds of problems I had to implement the patches to don't reuse "default" worker threads pool.
See attached HttpConduit.java patch how to achieve that.
Could you apply the patch for all three CXF souce files where "default" worker threads pool is reused?
The solution is to use e.g. Executors.newSingleThreadExecutor() instead of WorkQueueManagerImpl.getAutomaticWorkQueue() method.
Thanks,
JBossWS Team
PS: BTW, it took many hours of heavy debugging to identify this issue 