Tapestry
  1. Tapestry
  2. TAPESTRY-2561

Rapidly refreshing a page, even the same page, can cause a deadlock related to class loading

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 5.0.14
    • Fix Version/s: 5.0.15
    • Component/s: tapestry-core
    • Labels:
      None
    • Environment:
      Mac pro, OS Version: Mac OS X 10.5.4 (9E17)
      Java HotSpot(TM) Client VM (1.5.0_13-119 mixed mode,
      sharing)

      Description

      It was triggered by (accidentally) double clicking a pagelink
      Unfortunately, can't quite repeat it... at least not clicking with a mouse...

      1. full thread dump deadlock.txt
        90 kB
        Steven Woolley
      2. thread dump.txt
        71 kB
        Henning Petersen

        Issue Links

          Activity

          Hide
          Joe Hart added a comment -

          just upgraded to 5.0.15 and we are still getting this deadlock. Stack trace:

          Found one Java-level deadlock:
          =============================
          "http-8080-Processor9":
          waiting to lock monitor 0x8ecca804 (object 0xb38778f8, a java.lang.Class),
          which is held by "http-8080-Processor8"
          "http-8080-Processor8":
          waiting to lock monitor 0x087c3874 (object 0x9735d630, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader),
          which is held by "http-8080-Processor9"

          Java stack information for the threads listed above:
          ===================================================
          "http-8080-Processor9":
          at javassist.ClassPool.toClass2(ClassPool.java:992)

          • waiting to lock <0xb38778f8> (a java.lang.Class for javassist.ClassPool)
            at javassist.ClassPool.toClass(ClassPool.java:975)
            at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92)
            at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxyInstance(ModuleImpl.java:403)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxy(ModuleImpl.java:366)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.create(ModuleImpl.java:248)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.access$100(ModuleImpl.java:35)
            at org.apache.tapestry5.ioc.internal.ModuleImpl$1.invoke(ModuleImpl.java:168)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withWrite(ConcurrentBarrier.java:138)
            at org.apache.tapestry5.ioc.internal.ModuleImpl$2.invoke(ModuleImpl.java:185)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.findOrCreate(ModuleImpl.java:192)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.getService(ModuleImpl.java:89)
            at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:303)
            at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:527)
            at org.apache.tapestry5.ioc.internal.ObjectLocatorImpl.getService(ObjectLocatorImpl.java:44)
            at org.apache.tapestry5.internal.services.ServiceInjectionProvider.provideInjection(ServiceInjectionProvider.java:40)
            at $InjectionProvider_11d78158682.provideInjection($InjectionProvider_11d78158682.java)
            at $InjectionProvider_11d7815867a.provideInjection($InjectionProvider_11d7815867a.java)
            at org.apache.tapestry5.internal.transform.InjectWorker.transform(InjectWorker.java:57)
            at $ComponentClassTransformWorker_11d78158680.transform($ComponentClassTransformWorker_11d78158680.java)
            at $ComponentClassTransformWorker_11d78158676.transform($ComponentClassTransformWorker_11d78158676.java)
            at org.apache.tapestry5.internal.services.ComponentClassTransformerImpl.transformComponentClass(ComponentClassTransformerImpl.java:169)
            at $ComponentClassTransformer_11d7815861d.transformComponentClass($ComponentClassTransformer_11d7815861d.java)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:201)
            at javassist.Loader.findClass(Loader.java:340)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:92)
            at javassist.Loader.loadClass(Loader.java:311)
          • locked <0xb44ee148> (a java.lang.String)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
            at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
          • locked <0x9735d630> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at $Instantiator_11d781587e0.newInstance($Instantiator_11d781587e0.java)
            at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.<init>(InternalComponentResourcesImpl.java:83)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.<init>(ComponentPageElementImpl.java:584)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.newChild(ComponentPageElementImpl.java:518)
            at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newComponentElement(PageElementFactoryImpl.java:224)
            at $PageElementFactory_11d7815866b.newComponentElement($PageElementFactory_11d7815866b.java)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.startComponent(PageLoaderProcessor.java:664)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:509)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:815)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:401)
            at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59)
            at $PageLoader_11d78158669.loadPage($PageLoader_11d78158669.java)
            at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210)
            at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107)
            at $PagePool_11d78158668.checkout($PagePool_11d78158668.java)
            at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43)
            at $RequestPageCache_11d78158667.get($RequestPageCache_11d78158667.java)
            at $RequestPageCache_11d78158650.get($RequestPageCache_11d78158650.java)
            at lithium.lia.components.tapestry.services.layout.ComponentContributionSourceImpl.getComponentDescriptor(ComponentContributionSourceImpl.java:112)
            at $ComponentContributionSource_11d781586ca.getComponentDescriptor($ComponentContributionSource_11d781586ca.java)
            at lithium.lia.components.tapestry.components.layout.ComponentDisplay.beginRender(ComponentDisplay.java:49)
            at lithium.lia.components.tapestry.components.layout.ComponentDisplay.beginRender(ComponentDisplay.java)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11$1.run(ComponentPageElementImpl.java:347)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.invoke(ComponentPageElementImpl.java:918)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.access$300(ComponentPageElementImpl.java:50)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11.render(ComponentPageElementImpl.java:351)
            at org.apache.tapestry5.internal.services.RenderQueueImpl.run(RenderQueueImpl.java:72)
            at org.apache.tapestry5.internal.services.PageRenderQueueImpl.render(PageRenderQueueImpl.java:108)
            at $PageRenderQueue_11d781587b5.render($PageRenderQueue_11d781587b5.java)
            at $PageRenderQueue_11d781587a0.render($PageRenderQueue_11d781587a0.java)
            at org.apache.tapestry5.services.TapestryModule$19.renderMarkup(TapestryModule.java:1199)
            at lithium.web2.services.request.LithiumRenderSupport.renderMarkup(LithiumRenderSupport.java:51)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at lithium.web2.services.parameter.PageParameterProvider.renderMarkup(PageParameterProvider.java:34)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at lithium.web2.services.scripts.InjectStandardJavaScriptCommand.renderMarkup(InjectStandardJavaScriptCommand.java:48)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at org.apache.tapestry5.services.TapestryModule$28.renderMarkup(TapestryModule.java:1550)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at org.apache.tapestry5.services.TapestryModule$27.renderMarkup(TapestryModule.java:1531)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at org.apache.tapestry5.services.TapestryModule$26.renderMarkup(TapestryModule.java:1513)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at org.apache.tapestry5.services.TapestryModule$25.renderMarkup(TapestryModule.java:1493)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at lithium.lia.components.tapestry.services.LithiumCoreTapestryModule$2.renderMarkup(LithiumCoreTapestryModule.java:171)
            at $MarkupRendererFilter_11d781587b7.renderMarkup($MarkupRendererFilter_11d781587b7.java)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at lithium.lia.components.tapestry.services.LithiumCoreTapestryModule$1.renderMarkup(LithiumCoreTapestryModule.java:155)
            at $MarkupRendererFilter_11d781587b6.renderMarkup($MarkupRendererFilter_11d781587b6.java)
            at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java)
            at $MarkupRenderer_11d781587b4.renderMarkup($MarkupRenderer_11d781587b4.java)
            at org.apache.tapestry5.internal.services.PageMarkupRendererImpl.renderPageMarkup(PageMarkupRendererImpl.java:64)
            at $PageMarkupRenderer_11d781587b2.renderPageMarkup($PageMarkupRenderer_11d781587b2.java)
            at org.apache.tapestry5.internal.services.PageResponseRendererImpl.renderPageResponse(PageResponseRendererImpl.java:57)
            at $PageResponseRenderer_11d78158652.renderPageResponse($PageResponseRenderer_11d78158652.java)
            at org.apache.tapestry5.internal.services.PageRenderRequestHandlerImpl.handle(PageRenderRequestHandlerImpl.java:59)
            at lithium.web2.services.request.PageInitializationPageRenderRequestFilter.handle(PageInitializationPageRenderRequestFilter.java:72)
            at $PageRenderRequestHandler_11d78158653.handle($PageRenderRequestHandler_11d78158653.java)
            at org.apache.tapestry5.services.TapestryModule$33.handle(TapestryModule.java:1731)
            at $PageRenderRequestHandler_11d78158653.handle($PageRenderRequestHandler_11d78158653.java)
            at lithium.web2.services.session.SessionLastBoardRequestHandler.handle(SessionLastBoardRequestHandler.java:58)
            at lithium.web2.services.skins.SkinSetupPageRenderRequestHandler.handle(SkinSetupPageRenderRequestHandler.java:62)
            at lithium.web2.services.metrics.MetricsPageRenderRequestHandler.handle(MetricsPageRenderRequestHandler.java:110)
            at lithium.web2.services.text.LocaleProviderPageRenderRequestHandler.handle(LocaleProviderPageRenderRequestHandler.java:86)
            at lithium.web2.services.corenode.CoreNodeProviderPageRenderRequestHandler.handle(CoreNodeProviderPageRenderRequestHandler.java:52)
            at lithium.web2.services.navigation.RequestHistoryPageRenderRequestHandler.handle(RequestHistoryPageRenderRequestHandler.java:72)
            at lithium.lia.components.tapestry.services.parameter.ParameterMapServicePageRenderRequestHandler.handle(ParameterMapServicePageRenderRequestHandler.java:58)
            at lithium.lia.components.tapestry.services.request.ClearSessionRequestHandler.handle(ClearSessionRequestHandler.java:51)
            at lithium.lia.components.tapestry.services.request.RedirectPageRenderRequestHandler.handle(RedirectPageRenderRequestHandler.java:50)
            at lithium.lia.components.tapestry.services.page.CurrentPagePageRenderRequestHandlerDecorator.handle(CurrentPagePageRenderRequestHandlerDecorator.java:43)
            at lithium.web2.services.exceptions.ExceptionPageRenderRequestHandlerDecorator.handle(ExceptionPageRenderRequestHandlerDecorator.java:53)
            at $PageRenderRequestHandler_11d78158647.handle($PageRenderRequestHandler_11d78158647.java)
            at org.apache.tapestry5.internal.services.PageRenderDispatcher.process(PageRenderDispatcher.java:97)
            at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:73)
            at $Dispatcher_11d7815864d.dispatch($Dispatcher_11d7815864d.java)
            at $Dispatcher_11d7815863b.dispatch($Dispatcher_11d7815863b.java)
            at org.apache.tapestry5.services.TapestryModule$17.service(TapestryModule.java:1025)
            at lithium.web2.services.seourl.SeoRequestFilter.service(SeoRequestFilter.java:58)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.LocalizationFilter.service(LocalizationFilter.java:42)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:84)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at $RequestHandler_11d78158634.service($RequestHandler_11d78158634.java)
            at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005)
            at org.apache.tapestry5.upload.internal.services.MultipartServletRequestFilter.service(MultipartServletRequestFilter.java:44)
            at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java)
            at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62)
            at $HttpServletRequestFilter_11d78158632.service($HttpServletRequestFilter_11d78158632.java)
            at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java)
            at lithium.web2.services.request.LithiumRequestContextHttpServletRequestFilter.service(LithiumRequestContextHttpServletRequestFilter.java:50)
            at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java)
            at lithium.web2.services.exceptions.ServletRequestExceptionHandler.service(ServletRequestExceptionHandler.java:45)
            at $HttpServletRequestHandler_11d78158624.service($HttpServletRequestHandler_11d78158624.java)
            at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:179)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.limit.LimitFilter.doHttpFilter(LimitFilter.java:68)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.banuser.BannedUserFilter.doHttpFilter(BannedUserFilter.java:107)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.session.UserSessionFilter.doHttpFilter(UserSessionFilter.java:154)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.HttpRequestContextFilter.doHttpFilter(HttpRequestContextFilter.java:42)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.MimeFilter.doHttpFilter(MimeFilter.java:36)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.RequestTransformFilter.doHttpFilter(RequestTransformFilter.java:184)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.BlackboxFilter.doHttpFilter(BlackboxFilter.java:62)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.TrackingFilter$TrackingManager.doHttpFilter(TrackingFilter.java:160)
            at lithium.servlet.TrackingFilter.doHttpFilter(TrackingFilter.java:239)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.util.http.robots.RobotsFilter.doHttpFilter(RobotsFilter.java:47)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.MetricsFilter.doHttpFilter(MetricsFilter.java:40)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.PlaybackOriginalIpFilter.doHttpFilter(PlaybackOriginalIpFilter.java:53)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.session.ReplicatedSessionFilter.doFilter(ReplicatedSessionFilter.java:44)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.boards.servlet.ClearStateFilter.doHttpFilter(ClearStateFilter.java:34)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.rewrite.RewriteFilter.doHttpFilter(RewriteFilter.java:154)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
            at lithium.apps.main.webserver.Tomcat55Bootstrap$1.invoke(Tomcat55Bootstrap.java:190)
            at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:152)
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
            at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
            at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
            at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
            at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
            at java.lang.Thread.run(Thread.java:619)
            "http-8080-Processor8":
            at java.lang.ClassLoader.checkCerts(ClassLoader.java:751)
          • waiting to lock <0x9735d630> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487)
            at java.lang.ClassLoader.defineClass(ClassLoader.java:614)
            at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at javassist.ClassPool.toClass2(ClassPool.java:993)
          • locked <0xb38778f8> (a java.lang.Class for javassist.ClassPool)
            at javassist.ClassPool.toClass(ClassPool.java:975)
            at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92)
            at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.build(PropertyConduitSourceImpl.java:166)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.create(PropertyConduitSourceImpl.java:102)
            at $PropertyConduitSource_11d78158697.create($PropertyConduitSource_11d78158697.java)
            at org.apache.tapestry5.internal.bindings.PropBindingFactory.newBinding(PropBindingFactory.java:45)
            at $BindingFactory_11d78158698.newBinding($BindingFactory_11d78158698.java)
            at $BindingFactory_11d7815868e.newBinding($BindingFactory_11d7815868e.java)
            at org.apache.tapestry5.internal.services.BindingSourceImpl.newBinding(BindingSourceImpl.java:78)
            at $BindingSource_11d78158674.newBinding($BindingSource_11d78158674.java)
            at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newBinding(PageElementFactoryImpl.java:304)
            at $PageElementFactory_11d7815866b.newBinding($PageElementFactory_11d7815866b.java)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.findBinding(PageLoaderProcessor.java:270)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.bindParameterFromTemplate(PageLoaderProcessor.java:176)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.attribute(PageLoaderProcessor.java:305)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:513)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:815)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:401)
            at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59)
            at $PageLoader_11d78158669.loadPage($PageLoader_11d78158669.java)
            at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210)
            at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107)
            at $PagePool_11d78158668.checkout($PagePool_11d78158668.java)
            at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43)
            at $RequestPageCache_11d78158667.get($RequestPageCache_11d78158667.java)
            at $RequestPageCache_11d78158650.get($RequestPageCache_11d78158650.java)
            at lithium.lia.components.tapestry.services.page.CurrentRequestPageProviderImpl.get(CurrentRequestPageProviderImpl.java:43)
            at lithium.lia.components.tapestry.services.page.CurrentRequestPageProviderImpl.get(CurrentRequestPageProviderImpl.java:1)
            at $CurrentRequestPageProvider_11d78158666.get($CurrentRequestPageProvider_11d78158666.java)
            at lithium.lia.components.tapestry.services.page.CurrentRequestPageComponentProviderImpl.get(CurrentRequestPageComponentProviderImpl.java:42)
            at lithium.lia.components.tapestry.services.page.CurrentRequestPageComponentProviderImpl.get(CurrentRequestPageComponentProviderImpl.java:1)
            at $CurrentRequestPageComponentProvider_11d7815865c.get($CurrentRequestPageComponentProvider_11d7815865c.java)
            at lithium.lia.components.tapestry.services.request.ClearSessionRequestHandler.handle(ClearSessionRequestHandler.java:48)
            at lithium.lia.components.tapestry.services.request.RedirectPageRenderRequestHandler.handle(RedirectPageRenderRequestHandler.java:50)
            at lithium.lia.components.tapestry.services.page.CurrentPagePageRenderRequestHandlerDecorator.handle(CurrentPagePageRenderRequestHandlerDecorator.java:43)
            at lithium.web2.services.exceptions.ExceptionPageRenderRequestHandlerDecorator.handle(ExceptionPageRenderRequestHandlerDecorator.java:53)
            at $PageRenderRequestHandler_11d78158647.handle($PageRenderRequestHandler_11d78158647.java)
            at org.apache.tapestry5.internal.services.PageRenderDispatcher.process(PageRenderDispatcher.java:97)
            at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:73)
            at $Dispatcher_11d7815864d.dispatch($Dispatcher_11d7815864d.java)
            at $Dispatcher_11d7815863b.dispatch($Dispatcher_11d7815863b.java)
            at org.apache.tapestry5.services.TapestryModule$17.service(TapestryModule.java:1025)
            at lithium.web2.services.seourl.SeoRequestFilter.service(SeoRequestFilter.java:58)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.LocalizationFilter.service(LocalizationFilter.java:42)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:84)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106)
            at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java)
            at $RequestHandler_11d78158634.service($RequestHandler_11d78158634.java)
            at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005)
            at org.apache.tapestry5.upload.internal.services.MultipartServletRequestFilter.service(MultipartServletRequestFilter.java:44)
            at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java)
            at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62)
            at $HttpServletRequestFilter_11d78158632.service($HttpServletRequestFilter_11d78158632.java)
            at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java)
            at lithium.web2.services.request.LithiumRequestContextHttpServletRequestFilter.service(LithiumRequestContextHttpServletRequestFilter.java:50)
            at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java)
            at lithium.web2.services.exceptions.ServletRequestExceptionHandler.service(ServletRequestExceptionHandler.java:45)
            at $HttpServletRequestHandler_11d78158624.service($HttpServletRequestHandler_11d78158624.java)
            at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:179)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.limit.LimitFilter.doHttpFilter(LimitFilter.java:68)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.banuser.BannedUserFilter.doHttpFilter(BannedUserFilter.java:107)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.session.UserSessionFilter.doHttpFilter(UserSessionFilter.java:154)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.HttpRequestContextFilter.doHttpFilter(HttpRequestContextFilter.java:42)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.MimeFilter.doHttpFilter(MimeFilter.java:36)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.RequestTransformFilter.doHttpFilter(RequestTransformFilter.java:184)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.BlackboxFilter.doHttpFilter(BlackboxFilter.java:62)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.TrackingFilter$TrackingManager.doHttpFilter(TrackingFilter.java:160)
            at lithium.servlet.TrackingFilter.doHttpFilter(TrackingFilter.java:239)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.util.http.robots.RobotsFilter.doHttpFilter(RobotsFilter.java:47)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.MetricsFilter.doHttpFilter(MetricsFilter.java:40)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.PlaybackOriginalIpFilter.doHttpFilter(PlaybackOriginalIpFilter.java:53)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.session.ReplicatedSessionFilter.doFilter(ReplicatedSessionFilter.java:44)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.boards.servlet.ClearStateFilter.doHttpFilter(ClearStateFilter.java:34)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at lithium.servlet.rewrite.RewriteFilter.doHttpFilter(RewriteFilter.java:154)
            at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65)
            at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
            at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
            at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
            at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
            at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
            at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
            at lithium.apps.main.webserver.Tomcat55Bootstrap$1.invoke(Tomcat55Bootstrap.java:190)
            at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542)
            at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
            at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:152)
            at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
            at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
            at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
            at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
            at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
            at java.lang.Thread.run(Thread.java:619)

          Found 1 deadlock.

          Show
          Joe Hart added a comment - just upgraded to 5.0.15 and we are still getting this deadlock. Stack trace: Found one Java-level deadlock: ============================= "http-8080-Processor9": waiting to lock monitor 0x8ecca804 (object 0xb38778f8, a java.lang.Class), which is held by "http-8080-Processor8" "http-8080-Processor8": waiting to lock monitor 0x087c3874 (object 0x9735d630, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader), which is held by "http-8080-Processor9" Java stack information for the threads listed above: =================================================== "http-8080-Processor9": at javassist.ClassPool.toClass2(ClassPool.java:992) waiting to lock <0xb38778f8> (a java.lang.Class for javassist.ClassPool) at javassist.ClassPool.toClass(ClassPool.java:975) at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92) at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89) at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxyInstance(ModuleImpl.java:403) at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxy(ModuleImpl.java:366) at org.apache.tapestry5.ioc.internal.ModuleImpl.create(ModuleImpl.java:248) at org.apache.tapestry5.ioc.internal.ModuleImpl.access$100(ModuleImpl.java:35) at org.apache.tapestry5.ioc.internal.ModuleImpl$1.invoke(ModuleImpl.java:168) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withWrite(ConcurrentBarrier.java:138) at org.apache.tapestry5.ioc.internal.ModuleImpl$2.invoke(ModuleImpl.java:185) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83) at org.apache.tapestry5.ioc.internal.ModuleImpl.findOrCreate(ModuleImpl.java:192) at org.apache.tapestry5.ioc.internal.ModuleImpl.getService(ModuleImpl.java:89) at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:303) at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:527) at org.apache.tapestry5.ioc.internal.ObjectLocatorImpl.getService(ObjectLocatorImpl.java:44) at org.apache.tapestry5.internal.services.ServiceInjectionProvider.provideInjection(ServiceInjectionProvider.java:40) at $InjectionProvider_11d78158682.provideInjection($InjectionProvider_11d78158682.java) at $InjectionProvider_11d7815867a.provideInjection($InjectionProvider_11d7815867a.java) at org.apache.tapestry5.internal.transform.InjectWorker.transform(InjectWorker.java:57) at $ComponentClassTransformWorker_11d78158680.transform($ComponentClassTransformWorker_11d78158680.java) at $ComponentClassTransformWorker_11d78158676.transform($ComponentClassTransformWorker_11d78158676.java) at org.apache.tapestry5.internal.services.ComponentClassTransformerImpl.transformComponentClass(ComponentClassTransformerImpl.java:169) at $ComponentClassTransformer_11d7815861d.transformComponentClass($ComponentClassTransformer_11d7815861d.java) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:201) at javassist.Loader.findClass(Loader.java:340) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:92) at javassist.Loader.loadClass(Loader.java:311) locked <0xb44ee148> (a java.lang.String) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) locked <0x9735d630> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at $Instantiator_11d781587e0.newInstance($Instantiator_11d781587e0.java) at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.<init>(InternalComponentResourcesImpl.java:83) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.<init>(ComponentPageElementImpl.java:584) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.newChild(ComponentPageElementImpl.java:518) at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newComponentElement(PageElementFactoryImpl.java:224) at $PageElementFactory_11d7815866b.newComponentElement($PageElementFactory_11d7815866b.java) at org.apache.tapestry5.internal.services.PageLoaderProcessor.startComponent(PageLoaderProcessor.java:664) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:509) at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:815) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:401) at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59) at $PageLoader_11d78158669.loadPage($PageLoader_11d78158669.java) at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210) at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107) at $PagePool_11d78158668.checkout($PagePool_11d78158668.java) at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43) at $RequestPageCache_11d78158667.get($RequestPageCache_11d78158667.java) at $RequestPageCache_11d78158650.get($RequestPageCache_11d78158650.java) at lithium.lia.components.tapestry.services.layout.ComponentContributionSourceImpl.getComponentDescriptor(ComponentContributionSourceImpl.java:112) at $ComponentContributionSource_11d781586ca.getComponentDescriptor($ComponentContributionSource_11d781586ca.java) at lithium.lia.components.tapestry.components.layout.ComponentDisplay.beginRender(ComponentDisplay.java:49) at lithium.lia.components.tapestry.components.layout.ComponentDisplay.beginRender(ComponentDisplay.java) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11$1.run(ComponentPageElementImpl.java:347) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.invoke(ComponentPageElementImpl.java:918) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.access$300(ComponentPageElementImpl.java:50) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11.render(ComponentPageElementImpl.java:351) at org.apache.tapestry5.internal.services.RenderQueueImpl.run(RenderQueueImpl.java:72) at org.apache.tapestry5.internal.services.PageRenderQueueImpl.render(PageRenderQueueImpl.java:108) at $PageRenderQueue_11d781587b5.render($PageRenderQueue_11d781587b5.java) at $PageRenderQueue_11d781587a0.render($PageRenderQueue_11d781587a0.java) at org.apache.tapestry5.services.TapestryModule$19.renderMarkup(TapestryModule.java:1199) at lithium.web2.services.request.LithiumRenderSupport.renderMarkup(LithiumRenderSupport.java:51) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at lithium.web2.services.parameter.PageParameterProvider.renderMarkup(PageParameterProvider.java:34) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at lithium.web2.services.scripts.InjectStandardJavaScriptCommand.renderMarkup(InjectStandardJavaScriptCommand.java:48) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at org.apache.tapestry5.services.TapestryModule$28.renderMarkup(TapestryModule.java:1550) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at org.apache.tapestry5.services.TapestryModule$27.renderMarkup(TapestryModule.java:1531) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at org.apache.tapestry5.services.TapestryModule$26.renderMarkup(TapestryModule.java:1513) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at org.apache.tapestry5.services.TapestryModule$25.renderMarkup(TapestryModule.java:1493) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at lithium.lia.components.tapestry.services.LithiumCoreTapestryModule$2.renderMarkup(LithiumCoreTapestryModule.java:171) at $MarkupRendererFilter_11d781587b7.renderMarkup($MarkupRendererFilter_11d781587b7.java) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at lithium.lia.components.tapestry.services.LithiumCoreTapestryModule$1.renderMarkup(LithiumCoreTapestryModule.java:155) at $MarkupRendererFilter_11d781587b6.renderMarkup($MarkupRendererFilter_11d781587b6.java) at $MarkupRenderer_11d781587ba.renderMarkup($MarkupRenderer_11d781587ba.java) at $MarkupRenderer_11d781587b4.renderMarkup($MarkupRenderer_11d781587b4.java) at org.apache.tapestry5.internal.services.PageMarkupRendererImpl.renderPageMarkup(PageMarkupRendererImpl.java:64) at $PageMarkupRenderer_11d781587b2.renderPageMarkup($PageMarkupRenderer_11d781587b2.java) at org.apache.tapestry5.internal.services.PageResponseRendererImpl.renderPageResponse(PageResponseRendererImpl.java:57) at $PageResponseRenderer_11d78158652.renderPageResponse($PageResponseRenderer_11d78158652.java) at org.apache.tapestry5.internal.services.PageRenderRequestHandlerImpl.handle(PageRenderRequestHandlerImpl.java:59) at lithium.web2.services.request.PageInitializationPageRenderRequestFilter.handle(PageInitializationPageRenderRequestFilter.java:72) at $PageRenderRequestHandler_11d78158653.handle($PageRenderRequestHandler_11d78158653.java) at org.apache.tapestry5.services.TapestryModule$33.handle(TapestryModule.java:1731) at $PageRenderRequestHandler_11d78158653.handle($PageRenderRequestHandler_11d78158653.java) at lithium.web2.services.session.SessionLastBoardRequestHandler.handle(SessionLastBoardRequestHandler.java:58) at lithium.web2.services.skins.SkinSetupPageRenderRequestHandler.handle(SkinSetupPageRenderRequestHandler.java:62) at lithium.web2.services.metrics.MetricsPageRenderRequestHandler.handle(MetricsPageRenderRequestHandler.java:110) at lithium.web2.services.text.LocaleProviderPageRenderRequestHandler.handle(LocaleProviderPageRenderRequestHandler.java:86) at lithium.web2.services.corenode.CoreNodeProviderPageRenderRequestHandler.handle(CoreNodeProviderPageRenderRequestHandler.java:52) at lithium.web2.services.navigation.RequestHistoryPageRenderRequestHandler.handle(RequestHistoryPageRenderRequestHandler.java:72) at lithium.lia.components.tapestry.services.parameter.ParameterMapServicePageRenderRequestHandler.handle(ParameterMapServicePageRenderRequestHandler.java:58) at lithium.lia.components.tapestry.services.request.ClearSessionRequestHandler.handle(ClearSessionRequestHandler.java:51) at lithium.lia.components.tapestry.services.request.RedirectPageRenderRequestHandler.handle(RedirectPageRenderRequestHandler.java:50) at lithium.lia.components.tapestry.services.page.CurrentPagePageRenderRequestHandlerDecorator.handle(CurrentPagePageRenderRequestHandlerDecorator.java:43) at lithium.web2.services.exceptions.ExceptionPageRenderRequestHandlerDecorator.handle(ExceptionPageRenderRequestHandlerDecorator.java:53) at $PageRenderRequestHandler_11d78158647.handle($PageRenderRequestHandler_11d78158647.java) at org.apache.tapestry5.internal.services.PageRenderDispatcher.process(PageRenderDispatcher.java:97) at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:73) at $Dispatcher_11d7815864d.dispatch($Dispatcher_11d7815864d.java) at $Dispatcher_11d7815863b.dispatch($Dispatcher_11d7815863b.java) at org.apache.tapestry5.services.TapestryModule$17.service(TapestryModule.java:1025) at lithium.web2.services.seourl.SeoRequestFilter.service(SeoRequestFilter.java:58) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.LocalizationFilter.service(LocalizationFilter.java:42) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:84) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at $RequestHandler_11d78158634.service($RequestHandler_11d78158634.java) at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005) at org.apache.tapestry5.upload.internal.services.MultipartServletRequestFilter.service(MultipartServletRequestFilter.java:44) at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java) at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62) at $HttpServletRequestFilter_11d78158632.service($HttpServletRequestFilter_11d78158632.java) at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java) at lithium.web2.services.request.LithiumRequestContextHttpServletRequestFilter.service(LithiumRequestContextHttpServletRequestFilter.java:50) at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java) at lithium.web2.services.exceptions.ServletRequestExceptionHandler.service(ServletRequestExceptionHandler.java:45) at $HttpServletRequestHandler_11d78158624.service($HttpServletRequestHandler_11d78158624.java) at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:179) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.limit.LimitFilter.doHttpFilter(LimitFilter.java:68) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.banuser.BannedUserFilter.doHttpFilter(BannedUserFilter.java:107) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.session.UserSessionFilter.doHttpFilter(UserSessionFilter.java:154) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.HttpRequestContextFilter.doHttpFilter(HttpRequestContextFilter.java:42) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.MimeFilter.doHttpFilter(MimeFilter.java:36) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.RequestTransformFilter.doHttpFilter(RequestTransformFilter.java:184) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.BlackboxFilter.doHttpFilter(BlackboxFilter.java:62) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.TrackingFilter$TrackingManager.doHttpFilter(TrackingFilter.java:160) at lithium.servlet.TrackingFilter.doHttpFilter(TrackingFilter.java:239) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.util.http.robots.RobotsFilter.doHttpFilter(RobotsFilter.java:47) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.MetricsFilter.doHttpFilter(MetricsFilter.java:40) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.PlaybackOriginalIpFilter.doHttpFilter(PlaybackOriginalIpFilter.java:53) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.session.ReplicatedSessionFilter.doFilter(ReplicatedSessionFilter.java:44) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.boards.servlet.ClearStateFilter.doHttpFilter(ClearStateFilter.java:34) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.rewrite.RewriteFilter.doHttpFilter(RewriteFilter.java:154) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at lithium.apps.main.webserver.Tomcat55Bootstrap$1.invoke(Tomcat55Bootstrap.java:190) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:152) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:619) "http-8080-Processor8": at java.lang.ClassLoader.checkCerts(ClassLoader.java:751) waiting to lock <0x9735d630> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487) at java.lang.ClassLoader.defineClass(ClassLoader.java:614) at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at javassist.ClassPool.toClass2(ClassPool.java:993) locked <0xb38778f8> (a java.lang.Class for javassist.ClassPool) at javassist.ClassPool.toClass(ClassPool.java:975) at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92) at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.build(PropertyConduitSourceImpl.java:166) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.create(PropertyConduitSourceImpl.java:102) at $PropertyConduitSource_11d78158697.create($PropertyConduitSource_11d78158697.java) at org.apache.tapestry5.internal.bindings.PropBindingFactory.newBinding(PropBindingFactory.java:45) at $BindingFactory_11d78158698.newBinding($BindingFactory_11d78158698.java) at $BindingFactory_11d7815868e.newBinding($BindingFactory_11d7815868e.java) at org.apache.tapestry5.internal.services.BindingSourceImpl.newBinding(BindingSourceImpl.java:78) at $BindingSource_11d78158674.newBinding($BindingSource_11d78158674.java) at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newBinding(PageElementFactoryImpl.java:304) at $PageElementFactory_11d7815866b.newBinding($PageElementFactory_11d7815866b.java) at org.apache.tapestry5.internal.services.PageLoaderProcessor.findBinding(PageLoaderProcessor.java:270) at org.apache.tapestry5.internal.services.PageLoaderProcessor.bindParameterFromTemplate(PageLoaderProcessor.java:176) at org.apache.tapestry5.internal.services.PageLoaderProcessor.attribute(PageLoaderProcessor.java:305) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:513) at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:815) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:401) at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59) at $PageLoader_11d78158669.loadPage($PageLoader_11d78158669.java) at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210) at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107) at $PagePool_11d78158668.checkout($PagePool_11d78158668.java) at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43) at $RequestPageCache_11d78158667.get($RequestPageCache_11d78158667.java) at $RequestPageCache_11d78158650.get($RequestPageCache_11d78158650.java) at lithium.lia.components.tapestry.services.page.CurrentRequestPageProviderImpl.get(CurrentRequestPageProviderImpl.java:43) at lithium.lia.components.tapestry.services.page.CurrentRequestPageProviderImpl.get(CurrentRequestPageProviderImpl.java:1) at $CurrentRequestPageProvider_11d78158666.get($CurrentRequestPageProvider_11d78158666.java) at lithium.lia.components.tapestry.services.page.CurrentRequestPageComponentProviderImpl.get(CurrentRequestPageComponentProviderImpl.java:42) at lithium.lia.components.tapestry.services.page.CurrentRequestPageComponentProviderImpl.get(CurrentRequestPageComponentProviderImpl.java:1) at $CurrentRequestPageComponentProvider_11d7815865c.get($CurrentRequestPageComponentProvider_11d7815865c.java) at lithium.lia.components.tapestry.services.request.ClearSessionRequestHandler.handle(ClearSessionRequestHandler.java:48) at lithium.lia.components.tapestry.services.request.RedirectPageRenderRequestHandler.handle(RedirectPageRenderRequestHandler.java:50) at lithium.lia.components.tapestry.services.page.CurrentPagePageRenderRequestHandlerDecorator.handle(CurrentPagePageRenderRequestHandlerDecorator.java:43) at lithium.web2.services.exceptions.ExceptionPageRenderRequestHandlerDecorator.handle(ExceptionPageRenderRequestHandlerDecorator.java:53) at $PageRenderRequestHandler_11d78158647.handle($PageRenderRequestHandler_11d78158647.java) at org.apache.tapestry5.internal.services.PageRenderDispatcher.process(PageRenderDispatcher.java:97) at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:73) at $Dispatcher_11d7815864d.dispatch($Dispatcher_11d7815864d.java) at $Dispatcher_11d7815863b.dispatch($Dispatcher_11d7815863b.java) at org.apache.tapestry5.services.TapestryModule$17.service(TapestryModule.java:1025) at lithium.web2.services.seourl.SeoRequestFilter.service(SeoRequestFilter.java:58) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.LocalizationFilter.service(LocalizationFilter.java:42) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:84) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106) at $RequestHandler_11d7815863c.service($RequestHandler_11d7815863c.java) at $RequestHandler_11d78158634.service($RequestHandler_11d78158634.java) at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005) at org.apache.tapestry5.upload.internal.services.MultipartServletRequestFilter.service(MultipartServletRequestFilter.java:44) at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java) at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62) at $HttpServletRequestFilter_11d78158632.service($HttpServletRequestFilter_11d78158632.java) at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java) at lithium.web2.services.request.LithiumRequestContextHttpServletRequestFilter.service(LithiumRequestContextHttpServletRequestFilter.java:50) at $HttpServletRequestHandler_11d78158635.service($HttpServletRequestHandler_11d78158635.java) at lithium.web2.services.exceptions.ServletRequestExceptionHandler.service(ServletRequestExceptionHandler.java:45) at $HttpServletRequestHandler_11d78158624.service($HttpServletRequestHandler_11d78158624.java) at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:179) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.limit.LimitFilter.doHttpFilter(LimitFilter.java:68) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.banuser.BannedUserFilter.doHttpFilter(BannedUserFilter.java:107) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.session.UserSessionFilter.doHttpFilter(UserSessionFilter.java:154) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.HttpRequestContextFilter.doHttpFilter(HttpRequestContextFilter.java:42) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.MimeFilter.doHttpFilter(MimeFilter.java:36) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.RequestTransformFilter.doHttpFilter(RequestTransformFilter.java:184) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.BlackboxFilter.doHttpFilter(BlackboxFilter.java:62) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.TrackingFilter$TrackingManager.doHttpFilter(TrackingFilter.java:160) at lithium.servlet.TrackingFilter.doHttpFilter(TrackingFilter.java:239) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.util.http.robots.RobotsFilter.doHttpFilter(RobotsFilter.java:47) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.MetricsFilter.doHttpFilter(MetricsFilter.java:40) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.PlaybackOriginalIpFilter.doHttpFilter(PlaybackOriginalIpFilter.java:53) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.session.ReplicatedSessionFilter.doFilter(ReplicatedSessionFilter.java:44) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.boards.servlet.ClearStateFilter.doHttpFilter(ClearStateFilter.java:34) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at lithium.servlet.rewrite.RewriteFilter.doHttpFilter(RewriteFilter.java:154) at lithium.util.http.HttpFilter.doFilter(HttpFilter.java:65) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117) at lithium.apps.main.webserver.Tomcat55Bootstrap$1.invoke(Tomcat55Bootstrap.java:190) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:542) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:152) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870) at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665) at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528) at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81) at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685) at java.lang.Thread.run(Thread.java:619) Found 1 deadlock.
          Hide
          Henning Petersen added a comment -

          I still have the same problem in 5.0.15. If I don't warm up my app and click on links in my navigation randomly without giving the pages time to initialize, the application locks up. I've attached a thread dump. Everything halts in org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:73).

          Show
          Henning Petersen added a comment - I still have the same problem in 5.0.15. If I don't warm up my app and click on links in my navigation randomly without giving the pages time to initialize, the application locks up. I've attached a thread dump. Everything halts in org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:73).
          Hide
          jens breitenstein added a comment -

          currently this deadlock appears frequently when we start a tomcat under heavy load (let me guess in 95% of the time the deadlock occurs in 5% T5 starts correctly). Am I right, that the error occurs because the pages/components byte code manipulation is defered to first access of a particular page to minimize the startup time? If so what about just initializing all pages/components while starting up? ok, the startup time will increase but for production it plays no role if startup takes 30 sec or 5 min, right? this will be different for development I know, but when starting a TC locally there will be no heavy load so chances to get trapped by this error should be considerably low. Furthermore if T5 initializes all pages we will catch page errors immediately on startup instead of finding them "accidentally when clicking through the app" (but this is a different topic). Nevertheless I fear the frameset problem mentioned before by Jun Tsai is different from our's

          Show
          jens breitenstein added a comment - currently this deadlock appears frequently when we start a tomcat under heavy load (let me guess in 95% of the time the deadlock occurs in 5% T5 starts correctly). Am I right, that the error occurs because the pages/components byte code manipulation is defered to first access of a particular page to minimize the startup time? If so what about just initializing all pages/components while starting up? ok, the startup time will increase but for production it plays no role if startup takes 30 sec or 5 min, right? this will be different for development I know, but when starting a TC locally there will be no heavy load so chances to get trapped by this error should be considerably low. Furthermore if T5 initializes all pages we will catch page errors immediately on startup instead of finding them "accidentally when clicking through the app" (but this is a different topic). Nevertheless I fear the frameset problem mentioned before by Jun Tsai is different from our's
          Hide
          Jun Tsai added a comment -

          I have a frameset page.If I rapidly refresh the page,the deadlock occurs.

          Show
          Jun Tsai added a comment - I have a frameset page.If I rapidly refresh the page,the deadlock occurs.
          Hide
          Jun Tsai added a comment -

          I use r693314 code from subversion,The deadlock dump:

          Found one Java-level deadlock:
          =============================
          "qtp0-4":
          waiting to lock monitor 0x000000004046fe48 (object 0x00007ff51f072f58, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader),
          which is held by "qtp0-0"
          "qtp0-0":
          waiting to lock monitor 0x00000000403650d0 (object 0x00007ff4edf0c5e0, a java.lang.Class),
          which is held by "qtp0-4"

          Java stack information for the threads listed above:
          ===================================================
          "qtp0-4":
          at java.lang.ClassLoader.checkCerts(ClassLoader.java:751)

          • waiting to lock <0x00007ff51f072f58> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487)
            at java.lang.ClassLoader.defineClass(ClassLoader.java:614)
            at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at javassist.ClassPool.toClass2(ClassPool.java:965)
          • locked <0x00007ff4edf0c5e0> (a java.lang.Class for javassist.ClassPool)
            at javassist.ClassPool.toClass(ClassPool.java:947)
            at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92)
            at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.build(PropertyConduitSourceImpl.java:166)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.create(PropertyConduitSourceImpl.java:102)
            at $PropertyConduitSource_11c44822a5f.create($PropertyConduitSource_11c44822a5f.java)
            at org.apache.tapestry5.internal.bindings.PropBindingFactory.newBinding(PropBindingFactory.java:45)
            at $BindingFactory_11c44822a66.newBinding($BindingFactory_11c44822a66.java)
            at $BindingFactory_11c44822a46.newBinding($BindingFactory_11c44822a46.java)
            at org.apache.tapestry5.internal.services.BindingSourceImpl.newBinding(BindingSourceImpl.java:78)
            at $BindingSource_11c44822a33.newBinding($BindingSource_11c44822a33.java)
            at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newBinding(PageElementFactoryImpl.java:304)
            at $PageElementFactory_11c44822a2d.newBinding($PageElementFactory_11c44822a2d.java)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.findBinding(PageLoaderProcessor.java:265)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.bindParameterFromTemplate(PageLoaderProcessor.java:171)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.attribute(PageLoaderProcessor.java:300)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:505)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:808)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:393)
            at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59)
            at $PageLoader_11c44822a2b.loadPage($PageLoader_11c44822a2b.java)
            at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210)
            at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107)
            at $PagePool_11c44822a2a.checkout($PagePool_11c44822a2a.java)
            at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43)
            at $RequestPageCache_11c44822a29.get($RequestPageCache_11c44822a29.java)
            at $RequestPageCache_11c44822a21.get($RequestPageCache_11c44822a21.java)
            at org.apache.tapestry5.internal.services.DefaultRequestExceptionHandler.handleRequestException(DefaultRequestExceptionHandler.java:56)
            at $RequestExceptionHandler_11c44822a11.handleRequestException($RequestExceptionHandler_11c44822a11.java)
            at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:42)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:83)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at $RequestHandler_11c44822a09.service($RequestHandler_11c44822a09.java)
            at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005)
            at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62)
            at $HttpServletRequestFilter_11c44822a08.service($HttpServletRequestFilter_11c44822a08.java)
            at $HttpServletRequestHandler_11c44822a0a.service($HttpServletRequestHandler_11c44822a0a.java)
            at $HttpServletRequestHandler_11c44822a07.service($HttpServletRequestHandler_11c44822a07.java)
            at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:177)
            at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
            at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
            at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
            at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
            at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:757)
            at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:416)
            at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
            at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
            at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
            at org.mortbay.jetty.Server.handle(Server.java:324)
            at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
            at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:826)
            at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:523)
            at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
            at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:377)
            at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
            at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:497)
            "qtp0-0":
            at javassist.ClassPool.toClass2(ClassPool.java:964)
          • waiting to lock <0x00007ff4edf0c5e0> (a java.lang.Class for javassist.ClassPool)
            at javassist.ClassPool.toClass(ClassPool.java:947)
            at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92)
            at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxyInstance(ModuleImpl.java:385)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxy(ModuleImpl.java:348)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.create(ModuleImpl.java:230)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.access$000(ModuleImpl.java:35)
            at org.apache.tapestry5.ioc.internal.ModuleImpl$1.invoke(ModuleImpl.java:151)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withWrite(ConcurrentBarrier.java:138)
            at org.apache.tapestry5.ioc.internal.ModuleImpl$2.invoke(ModuleImpl.java:167)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.findOrCreate(ModuleImpl.java:174)
            at org.apache.tapestry5.ioc.internal.ModuleImpl.getService(ModuleImpl.java:82)
            at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:303)
            at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:527)
            at org.apache.tapestry5.ioc.internal.ObjectLocatorImpl.getService(ObjectLocatorImpl.java:44)
            at org.apache.tapestry5.internal.services.ServiceInjectionProvider.provideInjection(ServiceInjectionProvider.java:40)
            at $InjectionProvider_11c44822a5b.provideInjection($InjectionProvider_11c44822a5b.java)
            at $InjectionProvider_11c44822a3a.provideInjection($InjectionProvider_11c44822a3a.java)
            at org.apache.tapestry5.internal.transform.InjectWorker.transform(InjectWorker.java:57)
            at $ComponentClassTransformWorker_11c44822a40.transform($ComponentClassTransformWorker_11c44822a40.java)
            at $ComponentClassTransformWorker_11c44822a35.transform($ComponentClassTransformWorker_11c44822a35.java)
            at org.apache.tapestry5.internal.services.ComponentClassTransformerImpl.transformComponentClass(ComponentClassTransformerImpl.java:168)
            at $ComponentClassTransformer_11c44822a02.transformComponentClass($ComponentClassTransformer_11c44822a02.java)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:191)
            at javassist.Loader.findClass(Loader.java:340)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:90)
            at javassist.Loader.loadClass(Loader.java:311)
          • locked <0x00007ff4ee793370> (a java.lang.String)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.findClass(ComponentInstantiatorSourceImpl.java:279)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.forceSuperclassTransform(ComponentInstantiatorSourceImpl.java:239)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:187)
            at javassist.Loader.findClass(Loader.java:340)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:90)
            at javassist.Loader.loadClass(Loader.java:311)
          • locked <0x00007ff4ee792328> (a java.lang.String)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
            at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
          • locked <0x00007ff51f072f58> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at java.lang.Class.getDeclaredMethods0(Native Method)
            at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
            at java.lang.Class.getDeclaredMethods(Class.java:1791)
            at java.beans.Introspector$1.run(Introspector.java:1272)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.beans.Introspector.getPublicDeclaredMethods(Introspector.java:1270)
          • locked <0x00007ff4edeba640> (a java.lang.Class for java.beans.Introspector)
            at java.beans.Introspector.getTargetMethodInfo(Introspector.java:1136)
            at java.beans.Introspector.getBeanInfo(Introspector.java:387)
            at java.beans.Introspector.getBeanInfo(Introspector.java:159)
            at org.apache.tapestry5.ioc.internal.services.PropertyAccessImpl.buildAdapter(PropertyAccessImpl.java:84)
          • locked <0x00007ff5206b4aa8> (a org.apache.tapestry5.ioc.internal.services.PropertyAccessImpl)
            at org.apache.tapestry5.ioc.internal.services.PropertyAccessImpl.getAdapter(PropertyAccessImpl.java:65)
            at $PropertyAccess_11c448229f6.getAdapter($PropertyAccess_11c448229f6.java)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.infoForTerm(PropertyConduitSourceImpl.java:401)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.createNavigationMethod(PropertyConduitSourceImpl.java:292)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.build(PropertyConduitSourceImpl.java:156)
            at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.create(PropertyConduitSourceImpl.java:102)
            at $PropertyConduitSource_11c44822a5f.create($PropertyConduitSource_11c44822a5f.java)
            at org.apache.tapestry5.internal.bindings.PropBindingFactory.newBinding(PropBindingFactory.java:45)
            at $BindingFactory_11c44822a66.newBinding($BindingFactory_11c44822a66.java)
            at $BindingFactory_11c44822a46.newBinding($BindingFactory_11c44822a46.java)
            at org.apache.tapestry5.internal.services.BindingSourceImpl.newBinding(BindingSourceImpl.java:78)
            at $BindingSource_11c44822a33.newBinding($BindingSource_11c44822a33.java)
            at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newExpansionElement(PageElementFactoryImpl.java:176)
            at $PageElementFactory_11c44822a2d.newExpansionElement($PageElementFactory_11c44822a2d.java)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.expansion(PageLoaderProcessor.java:356)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:489)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:808)
            at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:393)
            at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59)
            at $PageLoader_11c44822a2b.loadPage($PageLoader_11c44822a2b.java)
            at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210)
            at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107)
            at $PagePool_11c44822a2a.checkout($PagePool_11c44822a2a.java)
            at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43)
            at $RequestPageCache_11c44822a29.get($RequestPageCache_11c44822a29.java)
            at $RequestPageCache_11c44822a21.get($RequestPageCache_11c44822a21.java)
            at org.apache.tapestry5.internal.services.LinkFactoryImpl.createPageLink(LinkFactoryImpl.java:274)
            at $LinkFactory_11c44822a25.createPageLink($LinkFactory_11c44822a25.java)
            at org.apache.tapestry5.internal.structure.PageImpl.createPageRenderLink(PageImpl.java:161)
            at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.createPageLink(InternalComponentResourcesImpl.java:145)
            at org.apache.tapestry5.corelib.components.PageLink.beginRender(PageLink.java:61)
            at org.apache.tapestry5.corelib.components.PageLink.beginRender(PageLink.java)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11$1.run(ComponentPageElementImpl.java:334)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.invoke(ComponentPageElementImpl.java:899)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.access$200(ComponentPageElementImpl.java:50)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11.render(ComponentPageElementImpl.java:338)
            at org.apache.tapestry5.internal.services.RenderQueueImpl.run(RenderQueueImpl.java:68)
            at org.apache.tapestry5.internal.services.PageRenderQueueImpl.render(PageRenderQueueImpl.java:108)
            at $PageRenderQueue_11c44822a52.render($PageRenderQueue_11c44822a52.java)
            at $PageRenderQueue_11c44822a50.render($PageRenderQueue_11c44822a50.java)
            at org.apache.tapestry5.services.TapestryModule$19.renderMarkup(TapestryModule.java:1199)
            at org.apache.tapestry5.services.TapestryModule$28.renderMarkup(TapestryModule.java:1550)
            at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java)
            at org.apache.tapestry5.services.TapestryModule$27.renderMarkup(TapestryModule.java:1531)
            at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java)
            at org.apache.tapestry5.services.TapestryModule$26.renderMarkup(TapestryModule.java:1513)
            at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java)
            at org.apache.tapestry5.services.TapestryModule$25.renderMarkup(TapestryModule.java:1493)
            at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java)
            at $MarkupRenderer_11c44822a4f.renderMarkup($MarkupRenderer_11c44822a4f.java)
            at org.apache.tapestry5.internal.services.PageMarkupRendererImpl.renderPageMarkup(PageMarkupRendererImpl.java:64)
            at $PageMarkupRenderer_11c44822a4d.renderPageMarkup($PageMarkupRenderer_11c44822a4d.java)
            at org.apache.tapestry5.internal.services.PageResponseRendererImpl.renderPageResponse(PageResponseRendererImpl.java:57)
            at $PageResponseRenderer_11c44822a23.renderPageResponse($PageResponseRenderer_11c44822a23.java)
            at org.apache.tapestry5.internal.services.PageRenderRequestHandlerImpl.handle(PageRenderRequestHandlerImpl.java:59)
            at org.apache.tapestry5.services.TapestryModule$33.handle(TapestryModule.java:1731)
            at $PageRenderRequestHandler_11c44822a24.handle($PageRenderRequestHandler_11c44822a24.java)
            at $PageRenderRequestHandler_11c44822a19.handle($PageRenderRequestHandler_11c44822a19.java)
            at org.apache.tapestry5.internal.services.PageRenderDispatcher.process(PageRenderDispatcher.java:97)
            at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:73)
            at $Dispatcher_11c44822a1f.dispatch($Dispatcher_11c44822a1f.java)
            at $Dispatcher_11c44822a12.dispatch($Dispatcher_11c44822a12.java)
            at org.apache.tapestry5.services.TapestryModule$17.service(TapestryModule.java:1025)
            at org.apache.tapestry5.internal.services.LocalizationFilter.service(LocalizationFilter.java:42)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:83)
            at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83)
            at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106)
            at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java)
            at $RequestHandler_11c44822a09.service($RequestHandler_11c44822a09.java)
            at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005)
            at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62)
            at $HttpServletRequestFilter_11c44822a08.service($HttpServletRequestFilter_11c44822a08.java)
            at $HttpServletRequestHandler_11c44822a0a.service($HttpServletRequestHandler_11c44822a0a.java)
            at $HttpServletRequestHandler_11c44822a07.service($HttpServletRequestHandler_11c44822a07.java)
            at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:177)
            at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115)
            at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361)
            at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
            at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
            at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:757)
            at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:416)
            at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206)
            at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
            at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
            at org.mortbay.jetty.Server.handle(Server.java:324)
            at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
            at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:826)
            at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:523)
            at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211)
            at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:377)
            at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
            at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:497)

          Found 1 deadlock.

          Show
          Jun Tsai added a comment - I use r693314 code from subversion,The deadlock dump: Found one Java-level deadlock: ============================= "qtp0-4": waiting to lock monitor 0x000000004046fe48 (object 0x00007ff51f072f58, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader), which is held by "qtp0-0" "qtp0-0": waiting to lock monitor 0x00000000403650d0 (object 0x00007ff4edf0c5e0, a java.lang.Class), which is held by "qtp0-4" Java stack information for the threads listed above: =================================================== "qtp0-4": at java.lang.ClassLoader.checkCerts(ClassLoader.java:751) waiting to lock <0x00007ff51f072f58> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487) at java.lang.ClassLoader.defineClass(ClassLoader.java:614) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at javassist.ClassPool.toClass2(ClassPool.java:965) locked <0x00007ff4edf0c5e0> (a java.lang.Class for javassist.ClassPool) at javassist.ClassPool.toClass(ClassPool.java:947) at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92) at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.build(PropertyConduitSourceImpl.java:166) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.create(PropertyConduitSourceImpl.java:102) at $PropertyConduitSource_11c44822a5f.create($PropertyConduitSource_11c44822a5f.java) at org.apache.tapestry5.internal.bindings.PropBindingFactory.newBinding(PropBindingFactory.java:45) at $BindingFactory_11c44822a66.newBinding($BindingFactory_11c44822a66.java) at $BindingFactory_11c44822a46.newBinding($BindingFactory_11c44822a46.java) at org.apache.tapestry5.internal.services.BindingSourceImpl.newBinding(BindingSourceImpl.java:78) at $BindingSource_11c44822a33.newBinding($BindingSource_11c44822a33.java) at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newBinding(PageElementFactoryImpl.java:304) at $PageElementFactory_11c44822a2d.newBinding($PageElementFactory_11c44822a2d.java) at org.apache.tapestry5.internal.services.PageLoaderProcessor.findBinding(PageLoaderProcessor.java:265) at org.apache.tapestry5.internal.services.PageLoaderProcessor.bindParameterFromTemplate(PageLoaderProcessor.java:171) at org.apache.tapestry5.internal.services.PageLoaderProcessor.attribute(PageLoaderProcessor.java:300) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:505) at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:808) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:393) at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59) at $PageLoader_11c44822a2b.loadPage($PageLoader_11c44822a2b.java) at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210) at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107) at $PagePool_11c44822a2a.checkout($PagePool_11c44822a2a.java) at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43) at $RequestPageCache_11c44822a29.get($RequestPageCache_11c44822a29.java) at $RequestPageCache_11c44822a21.get($RequestPageCache_11c44822a21.java) at org.apache.tapestry5.internal.services.DefaultRequestExceptionHandler.handleRequestException(DefaultRequestExceptionHandler.java:56) at $RequestExceptionHandler_11c44822a11.handleRequestException($RequestExceptionHandler_11c44822a11.java) at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:42) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:83) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at $RequestHandler_11c44822a09.service($RequestHandler_11c44822a09.java) at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005) at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62) at $HttpServletRequestFilter_11c44822a08.service($HttpServletRequestFilter_11c44822a08.java) at $HttpServletRequestHandler_11c44822a0a.service($HttpServletRequestHandler_11c44822a0a.java) at $HttpServletRequestHandler_11c44822a07.service($HttpServletRequestHandler_11c44822a07.java) at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:177) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:757) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:416) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:826) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:523) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:377) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:497) "qtp0-0": at javassist.ClassPool.toClass2(ClassPool.java:964) waiting to lock <0x00007ff4edf0c5e0> (a java.lang.Class for javassist.ClassPool) at javassist.ClassPool.toClass(ClassPool.java:947) at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:92) at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89) at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxyInstance(ModuleImpl.java:385) at org.apache.tapestry5.ioc.internal.ModuleImpl.createProxy(ModuleImpl.java:348) at org.apache.tapestry5.ioc.internal.ModuleImpl.create(ModuleImpl.java:230) at org.apache.tapestry5.ioc.internal.ModuleImpl.access$000(ModuleImpl.java:35) at org.apache.tapestry5.ioc.internal.ModuleImpl$1.invoke(ModuleImpl.java:151) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withWrite(ConcurrentBarrier.java:138) at org.apache.tapestry5.ioc.internal.ModuleImpl$2.invoke(ModuleImpl.java:167) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83) at org.apache.tapestry5.ioc.internal.ModuleImpl.findOrCreate(ModuleImpl.java:174) at org.apache.tapestry5.ioc.internal.ModuleImpl.getService(ModuleImpl.java:82) at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:303) at org.apache.tapestry5.ioc.internal.RegistryImpl.getService(RegistryImpl.java:527) at org.apache.tapestry5.ioc.internal.ObjectLocatorImpl.getService(ObjectLocatorImpl.java:44) at org.apache.tapestry5.internal.services.ServiceInjectionProvider.provideInjection(ServiceInjectionProvider.java:40) at $InjectionProvider_11c44822a5b.provideInjection($InjectionProvider_11c44822a5b.java) at $InjectionProvider_11c44822a3a.provideInjection($InjectionProvider_11c44822a3a.java) at org.apache.tapestry5.internal.transform.InjectWorker.transform(InjectWorker.java:57) at $ComponentClassTransformWorker_11c44822a40.transform($ComponentClassTransformWorker_11c44822a40.java) at $ComponentClassTransformWorker_11c44822a35.transform($ComponentClassTransformWorker_11c44822a35.java) at org.apache.tapestry5.internal.services.ComponentClassTransformerImpl.transformComponentClass(ComponentClassTransformerImpl.java:168) at $ComponentClassTransformer_11c44822a02.transformComponentClass($ComponentClassTransformer_11c44822a02.java) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:191) at javassist.Loader.findClass(Loader.java:340) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:90) at javassist.Loader.loadClass(Loader.java:311) locked <0x00007ff4ee793370> (a java.lang.String) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.findClass(ComponentInstantiatorSourceImpl.java:279) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.forceSuperclassTransform(ComponentInstantiatorSourceImpl.java:239) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:187) at javassist.Loader.findClass(Loader.java:340) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:90) at javassist.Loader.loadClass(Loader.java:311) locked <0x00007ff4ee792328> (a java.lang.String) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) locked <0x00007ff51f072f58> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2427) at java.lang.Class.getDeclaredMethods(Class.java:1791) at java.beans.Introspector$1.run(Introspector.java:1272) at java.security.AccessController.doPrivileged(Native Method) at java.beans.Introspector.getPublicDeclaredMethods(Introspector.java:1270) locked <0x00007ff4edeba640> (a java.lang.Class for java.beans.Introspector) at java.beans.Introspector.getTargetMethodInfo(Introspector.java:1136) at java.beans.Introspector.getBeanInfo(Introspector.java:387) at java.beans.Introspector.getBeanInfo(Introspector.java:159) at org.apache.tapestry5.ioc.internal.services.PropertyAccessImpl.buildAdapter(PropertyAccessImpl.java:84) locked <0x00007ff5206b4aa8> (a org.apache.tapestry5.ioc.internal.services.PropertyAccessImpl) at org.apache.tapestry5.ioc.internal.services.PropertyAccessImpl.getAdapter(PropertyAccessImpl.java:65) at $PropertyAccess_11c448229f6.getAdapter($PropertyAccess_11c448229f6.java) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.infoForTerm(PropertyConduitSourceImpl.java:401) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.createNavigationMethod(PropertyConduitSourceImpl.java:292) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.build(PropertyConduitSourceImpl.java:156) at org.apache.tapestry5.internal.services.PropertyConduitSourceImpl.create(PropertyConduitSourceImpl.java:102) at $PropertyConduitSource_11c44822a5f.create($PropertyConduitSource_11c44822a5f.java) at org.apache.tapestry5.internal.bindings.PropBindingFactory.newBinding(PropBindingFactory.java:45) at $BindingFactory_11c44822a66.newBinding($BindingFactory_11c44822a66.java) at $BindingFactory_11c44822a46.newBinding($BindingFactory_11c44822a46.java) at org.apache.tapestry5.internal.services.BindingSourceImpl.newBinding(BindingSourceImpl.java:78) at $BindingSource_11c44822a33.newBinding($BindingSource_11c44822a33.java) at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newExpansionElement(PageElementFactoryImpl.java:176) at $PageElementFactory_11c44822a2d.newExpansionElement($PageElementFactory_11c44822a2d.java) at org.apache.tapestry5.internal.services.PageLoaderProcessor.expansion(PageLoaderProcessor.java:356) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadTemplateForComponent(PageLoaderProcessor.java:489) at org.apache.tapestry5.internal.services.PageLoaderProcessor.workComponentQueue(PageLoaderProcessor.java:808) at org.apache.tapestry5.internal.services.PageLoaderProcessor.loadPage(PageLoaderProcessor.java:393) at org.apache.tapestry5.internal.services.PageLoaderImpl.loadPage(PageLoaderImpl.java:59) at $PageLoader_11c44822a2b.loadPage($PageLoader_11c44822a2b.java) at org.apache.tapestry5.internal.services.PagePoolCache.checkout(PagePoolCache.java:210) at org.apache.tapestry5.internal.services.PagePoolImpl.checkout(PagePoolImpl.java:107) at $PagePool_11c44822a2a.checkout($PagePool_11c44822a2a.java) at org.apache.tapestry5.internal.services.RequestPageCacheImpl.get(RequestPageCacheImpl.java:43) at $RequestPageCache_11c44822a29.get($RequestPageCache_11c44822a29.java) at $RequestPageCache_11c44822a21.get($RequestPageCache_11c44822a21.java) at org.apache.tapestry5.internal.services.LinkFactoryImpl.createPageLink(LinkFactoryImpl.java:274) at $LinkFactory_11c44822a25.createPageLink($LinkFactory_11c44822a25.java) at org.apache.tapestry5.internal.structure.PageImpl.createPageRenderLink(PageImpl.java:161) at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.createPageLink(InternalComponentResourcesImpl.java:145) at org.apache.tapestry5.corelib.components.PageLink.beginRender(PageLink.java:61) at org.apache.tapestry5.corelib.components.PageLink.beginRender(PageLink.java) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11$1.run(ComponentPageElementImpl.java:334) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.invoke(ComponentPageElementImpl.java:899) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.access$200(ComponentPageElementImpl.java:50) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl$11.render(ComponentPageElementImpl.java:338) at org.apache.tapestry5.internal.services.RenderQueueImpl.run(RenderQueueImpl.java:68) at org.apache.tapestry5.internal.services.PageRenderQueueImpl.render(PageRenderQueueImpl.java:108) at $PageRenderQueue_11c44822a52.render($PageRenderQueue_11c44822a52.java) at $PageRenderQueue_11c44822a50.render($PageRenderQueue_11c44822a50.java) at org.apache.tapestry5.services.TapestryModule$19.renderMarkup(TapestryModule.java:1199) at org.apache.tapestry5.services.TapestryModule$28.renderMarkup(TapestryModule.java:1550) at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java) at org.apache.tapestry5.services.TapestryModule$27.renderMarkup(TapestryModule.java:1531) at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java) at org.apache.tapestry5.services.TapestryModule$26.renderMarkup(TapestryModule.java:1513) at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java) at org.apache.tapestry5.services.TapestryModule$25.renderMarkup(TapestryModule.java:1493) at $MarkupRenderer_11c44822a57.renderMarkup($MarkupRenderer_11c44822a57.java) at $MarkupRenderer_11c44822a4f.renderMarkup($MarkupRenderer_11c44822a4f.java) at org.apache.tapestry5.internal.services.PageMarkupRendererImpl.renderPageMarkup(PageMarkupRendererImpl.java:64) at $PageMarkupRenderer_11c44822a4d.renderPageMarkup($PageMarkupRenderer_11c44822a4d.java) at org.apache.tapestry5.internal.services.PageResponseRendererImpl.renderPageResponse(PageResponseRendererImpl.java:57) at $PageResponseRenderer_11c44822a23.renderPageResponse($PageResponseRenderer_11c44822a23.java) at org.apache.tapestry5.internal.services.PageRenderRequestHandlerImpl.handle(PageRenderRequestHandlerImpl.java:59) at org.apache.tapestry5.services.TapestryModule$33.handle(TapestryModule.java:1731) at $PageRenderRequestHandler_11c44822a24.handle($PageRenderRequestHandler_11c44822a24.java) at $PageRenderRequestHandler_11c44822a19.handle($PageRenderRequestHandler_11c44822a19.java) at org.apache.tapestry5.internal.services.PageRenderDispatcher.process(PageRenderDispatcher.java:97) at org.apache.tapestry5.internal.services.PageRenderDispatcher.dispatch(PageRenderDispatcher.java:73) at $Dispatcher_11c44822a1f.dispatch($Dispatcher_11c44822a1f.java) at $Dispatcher_11c44822a12.dispatch($Dispatcher_11c44822a12.java) at org.apache.tapestry5.services.TapestryModule$17.service(TapestryModule.java:1025) at org.apache.tapestry5.internal.services.LocalizationFilter.service(LocalizationFilter.java:42) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.internal.services.RequestErrorFilter.service(RequestErrorFilter.java:26) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.services.TapestryModule$3.service(TapestryModule.java:624) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.services.TapestryModule$2.service(TapestryModule.java:614) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.internal.services.StaticFilesFilter.service(StaticFilesFilter.java:85) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:93) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter$2.invoke(CheckForUpdatesFilter.java:83) at org.apache.tapestry5.ioc.internal.util.ConcurrentBarrier.withRead(ConcurrentBarrier.java:83) at org.apache.tapestry5.internal.services.CheckForUpdatesFilter.service(CheckForUpdatesFilter.java:106) at $RequestHandler_11c44822a13.service($RequestHandler_11c44822a13.java) at $RequestHandler_11c44822a09.service($RequestHandler_11c44822a09.java) at org.apache.tapestry5.services.TapestryModule$16.service(TapestryModule.java:1005) at org.apache.tapestry5.internal.services.IgnoredPathsFilter.service(IgnoredPathsFilter.java:62) at $HttpServletRequestFilter_11c44822a08.service($HttpServletRequestFilter_11c44822a08.java) at $HttpServletRequestHandler_11c44822a0a.service($HttpServletRequestHandler_11c44822a0a.java) at $HttpServletRequestHandler_11c44822a07.service($HttpServletRequestHandler_11c44822a07.java) at org.apache.tapestry5.TapestryFilter.doFilter(TapestryFilter.java:177) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1115) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:361) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:757) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:416) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:206) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:324) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:826) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:523) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:377) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:497) Found 1 deadlock.
          Hide
          Howard M. Lewis Ship added a comment -

          I'm marking this closed again; if you hit a deadlock please reopen it and try to document the deadlock AND what was going on in the client and the server when the deadlock occurred.

          Show
          Howard M. Lewis Ship added a comment - I'm marking this closed again; if you hit a deadlock please reopen it and try to document the deadlock AND what was going on in the client and the server when the deadlock occurred.
          Hide
          Howard M. Lewis Ship added a comment -

          FYI: Ultimately, I think the problem here was the fact that javassist.ClassPool needs to invoke setAccessible() in order to invoke defineClass(), a protected ClassLoader method. That is, I believe, what ends up locking the class loader (though it's hard to tell, at it may be the actual call to defineClass() that does it).

          In any case, introducing the extra URLClassLoader between the context class loader and the component layer class loader is doing the trick.

          Show
          Howard M. Lewis Ship added a comment - FYI: Ultimately, I think the problem here was the fact that javassist.ClassPool needs to invoke setAccessible() in order to invoke defineClass(), a protected ClassLoader method. That is, I believe, what ends up locking the class loader (though it's hard to tell, at it may be the actual call to defineClass() that does it). In any case, introducing the extra URLClassLoader between the context class loader and the component layer class loader is doing the trick.
          Hide
          Howard M. Lewis Ship added a comment -

          Adding the "buffer" class loader seems to be doing the trick. That one change and all of the crazy extra synchronization on the context class loader goes away, taking with it (I hope) the deadlocks.

          I'll have this into SVN shortly; hopefully everyone can build from SVN and retest.

          Show
          Howard M. Lewis Ship added a comment - Adding the "buffer" class loader seems to be doing the trick. That one change and all of the crazy extra synchronization on the context class loader goes away, taking with it (I hope) the deadlocks. I'll have this into SVN shortly; hopefully everyone can build from SVN and retest.
          Hide
          Howard M. Lewis Ship added a comment -

          The problem is knowing what goes on inside certain native methods of ClassLoader.

          Show
          Howard M. Lewis Ship added a comment - The problem is knowing what goes on inside certain native methods of ClassLoader.
          Hide
          Howard M. Lewis Ship added a comment -

          I'm exploring another possiblility, introducing an "extra" class loader between the context class loader and the component class loader. This might remove the need for a lot of synchronization on the context class loader (which is the root of the problem).

          Show
          Howard M. Lewis Ship added a comment - I'm exploring another possiblility, introducing an "extra" class loader between the context class loader and the component class loader. This might remove the need for a lot of synchronization on the context class loader (which is the root of the problem).
          Hide
          Howard M. Lewis Ship added a comment -

          Looks like Jun Tsai has found a new deadlock.

          Jun — I really need that information about how you generate the deadlock.

          Show
          Howard M. Lewis Ship added a comment - Looks like Jun Tsai has found a new deadlock. Jun — I really need that information about how you generate the deadlock.
          Hide
          Jun Tsai added a comment -

          produce the new deadlock,

          tapestry: tapestry-ioc-5.0.15-20080908.110035-30.jar
          jetty:6.1.6
          os:ubuntu
          jdk:1.6.0_06

          Show
          Jun Tsai added a comment - produce the new deadlock, tapestry: tapestry-ioc-5.0.15-20080908.110035-30.jar jetty:6.1.6 os:ubuntu jdk:1.6.0_06
          Hide
          Jun Tsai added a comment -

          I had pulled the source from svn.I use mvn -Dmaven.test.skip=true clean install.

          Show
          Jun Tsai added a comment - I had pulled the source from svn.I use mvn -Dmaven.test.skip=true clean install.
          Hide
          Nicolas Charles added a comment -

          Using the same application as Francois Armand described earlier, i tried the 5.0.15-SNAPSHOT this morning, and the problem seems to be solved. I tried several bunches of stress tests using JMeter & Jetty, without any deadlock (using the 5.0.14 leads immediately to a deadlock)

          The configuration used is Linux Ubuntu, Jetty 6.1.6, jdk 1.6.0_06

          Show
          Nicolas Charles added a comment - Using the same application as Francois Armand described earlier, i tried the 5.0.15-SNAPSHOT this morning, and the problem seems to be solved. I tried several bunches of stress tests using JMeter & Jetty, without any deadlock (using the 5.0.14 leads immediately to a deadlock) The configuration used is Linux Ubuntu, Jetty 6.1.6, jdk 1.6.0_06
          Hide
          Filip S. Adamsen added a comment -

          Unless you pulled the code out of Subversion you're not using the snapshot Howard mentioned - it's not up yet.

          Show
          Filip S. Adamsen added a comment - Unless you pulled the code out of Subversion you're not using the snapshot Howard mentioned - it's not up yet.
          Hide
          Jun Tsai added a comment -

          I use the latest version.I find a new deadlock:

          Found one Java-level deadlock:
          =============================
          "btpool0-5":
          waiting to lock monitor 0x000000004012b298 (object 0x00007fe7e44887b0, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader),
          which is held by "btpool0-2"
          "btpool0-2":
          waiting to lock monitor 0x00007fe7b1c83d38 (object 0x00007fe7e44278c8, a org.mortbay.jetty.webapp.WebAppClassLoader),
          which is held by "btpool0-5"

          Java stack information for the threads listed above:
          ===================================================
          "btpool0-5":
          at java.lang.ClassLoader.getDefaultDomain(ClassLoader.java:1583)

          • waiting to lock <0x00007fe7e44887b0> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at java.lang.ClassLoader.preDefineClass(ClassLoader.java:483)
            at java.lang.ClassLoader.defineClass(ClassLoader.java:614)
            at java.lang.ClassLoader.defineClass(ClassLoader.java:465)
            at javassist.Loader.findClass(Loader.java:379)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:109)
            at javassist.Loader.loadClass(Loader.java:311)
          • locked <0x00007fe7b373c698> (a java.lang.String)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.loadClass(ComponentInstantiatorSourceImpl.java:86)
          • locked <0x00007fe7e44278c8> (a org.mortbay.jetty.webapp.WebAppClassLoader)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.findClass(ComponentInstantiatorSourceImpl.java:285)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.findInstantiator(ComponentInstantiatorSourceImpl.java:271)
          • locked <0x00007fe7e4484ac8> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl)
            at $ComponentInstantiatorSource_11c40261855.findInstantiator($ComponentInstantiatorSource_11c40261855.java)
            at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newRootComponentElement(PageElementFactoryImpl.java:257)

          ......
          "btpool0-2":
          at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.loadClass(ComponentInstantiatorSourceImpl.java:86)

          • waiting to lock <0x00007fe7e44278c8> (a org.mortbay.jetty.webapp.WebAppClassLoader)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
            at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
          • locked <0x00007fe7e44887b0> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at $Instantiator_11c402618c4.newInstance($Instantiator_11c402618c4.java)
            at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.<init>(InternalComponentResourcesImpl.java:83)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.<init>(ComponentPageElementImpl.java:571)
            at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.<init>(ComponentPageElementImpl.java:584)
            at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newRootComponentElement(PageElementFactoryImpl.java:261)
            ...............
          Show
          Jun Tsai added a comment - I use the latest version.I find a new deadlock: Found one Java-level deadlock: ============================= "btpool0-5": waiting to lock monitor 0x000000004012b298 (object 0x00007fe7e44887b0, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader), which is held by "btpool0-2" "btpool0-2": waiting to lock monitor 0x00007fe7b1c83d38 (object 0x00007fe7e44278c8, a org.mortbay.jetty.webapp.WebAppClassLoader), which is held by "btpool0-5" Java stack information for the threads listed above: =================================================== "btpool0-5": at java.lang.ClassLoader.getDefaultDomain(ClassLoader.java:1583) waiting to lock <0x00007fe7e44887b0> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at java.lang.ClassLoader.preDefineClass(ClassLoader.java:483) at java.lang.ClassLoader.defineClass(ClassLoader.java:614) at java.lang.ClassLoader.defineClass(ClassLoader.java:465) at javassist.Loader.findClass(Loader.java:379) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:109) at javassist.Loader.loadClass(Loader.java:311) locked <0x00007fe7b373c698> (a java.lang.String) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.loadClass(ComponentInstantiatorSourceImpl.java:86) locked <0x00007fe7e44278c8> (a org.mortbay.jetty.webapp.WebAppClassLoader) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.findClass(ComponentInstantiatorSourceImpl.java:285) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.findInstantiator(ComponentInstantiatorSourceImpl.java:271) locked <0x00007fe7e4484ac8> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl) at $ComponentInstantiatorSource_11c40261855.findInstantiator($ComponentInstantiatorSource_11c40261855.java) at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newRootComponentElement(PageElementFactoryImpl.java:257) ...... "btpool0-2": at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.loadClass(ComponentInstantiatorSourceImpl.java:86) waiting to lock <0x00007fe7e44278c8> (a org.mortbay.jetty.webapp.WebAppClassLoader) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) locked <0x00007fe7e44887b0> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at $Instantiator_11c402618c4.newInstance($Instantiator_11c402618c4.java) at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.<init>(InternalComponentResourcesImpl.java:83) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.<init>(ComponentPageElementImpl.java:571) at org.apache.tapestry5.internal.structure.ComponentPageElementImpl.<init>(ComponentPageElementImpl.java:584) at org.apache.tapestry5.internal.services.PageElementFactoryImpl.newRootComponentElement(PageElementFactoryImpl.java:261) ...............
          Hide
          Howard M. Lewis Ship added a comment -

          I think I've nailed it. The trick is to ensure that the context class loader's monitor is locked before the component class loader. Also, reduced the synchronization inside ModuleImpl, to alllow many readers/one writer when accessing service proxies.

          Those who have been affected by the bug should retest with the next snapshot (i.e., available on monday sep 8) to see if they can reproduce the problem.

          Show
          Howard M. Lewis Ship added a comment - I think I've nailed it. The trick is to ensure that the context class loader's monitor is locked before the component class loader. Also, reduced the synchronization inside ModuleImpl, to alllow many readers/one writer when accessing service proxies. Those who have been affected by the bug should retest with the next snapshot (i.e., available on monday sep 8) to see if they can reproduce the problem.
          Hide
          Howard M. Lewis Ship added a comment -

          Ouch. Have to turn off running of tests in parallel. Ordinary code (typical unit tests, PageTester, etc.) uses the normal application class loader. Selenium-based tests are using a Jetty-provided context class loader. Thus they synchronize on different CLs and hit deadlocks on other mutexes. Ouch!

          Show
          Howard M. Lewis Ship added a comment - Ouch. Have to turn off running of tests in parallel. Ordinary code (typical unit tests, PageTester, etc.) uses the normal application class loader. Selenium-based tests are using a Jetty-provided context class loader. Thus they synchronize on different CLs and hit deadlocks on other mutexes. Ouch!
          Hide
          Howard M. Lewis Ship added a comment -

          Definately wack-a-mole. The problem is that you need to always lock the context class loader, then lock the component class loader. Finding all the places where that can happen is painful, given that the Java runtime is constantly loading new classes as well. Tests, which use the class loaders differently than the normal production code, are also affected.

          Show
          Howard M. Lewis Ship added a comment - Definately wack-a-mole. The problem is that you need to always lock the context class loader, then lock the component class loader. Finding all the places where that can happen is painful, given that the Java runtime is constantly loading new classes as well. Tests, which use the class loaders differently than the normal production code, are also affected.
          Hide
          Howard M. Lewis Ship added a comment -

          First, I'm able to reliably reproduce a deadlock (hopefully, the only deadlock) using JMeter on my MacBook Pro. That's great, since I don't have to try to reproduce some of these environments.

          I've been running experiments and have found that the right solution is:

          1) The PackageAwareLoader (the loader for component classes) must synchronize its activity against the parent class loader (the context class loader).
          2) The global mutex should be eliminated, it just causes deadlocks.

          With these changes in place, I can no longer reproduce the deadlock.

          Show
          Howard M. Lewis Ship added a comment - First, I'm able to reliably reproduce a deadlock (hopefully, the only deadlock) using JMeter on my MacBook Pro. That's great, since I don't have to try to reproduce some of these environments. I've been running experiments and have found that the right solution is: 1) The PackageAwareLoader (the loader for component classes) must synchronize its activity against the parent class loader (the context class loader). 2) The global mutex should be eliminated, it just causes deadlocks. With these changes in place, I can no longer reproduce the deadlock.
          Hide
          Francois Armand added a comment -

          We have done some load test on our application, and one time, we get the deadlock, so I will try to explain what we did.

          • if I start the application, and use it a little (even if I don't go everywhere), everything is ok : on a load test, we were able to let the server run under the load of 4 computers, 25 jmeter threads by computer during several hours, without the deadlock.
          • if I start the application and immediately launch the jmeter test with say 25 threads, the application goes to the deadlock.

          I tried to reproduce this bug in a demo application, but did not succeed. It seems that trivial application run just fine, even with a bunch of thread launch at once.

          Show
          Francois Armand added a comment - We have done some load test on our application, and one time, we get the deadlock, so I will try to explain what we did. if I start the application, and use it a little (even if I don't go everywhere), everything is ok : on a load test, we were able to let the server run under the load of 4 computers, 25 jmeter threads by computer during several hours, without the deadlock. if I start the application and immediately launch the jmeter test with say 25 threads, the application goes to the deadlock. I tried to reproduce this bug in a demo application, but did not succeed. It seems that trivial application run just fine, even with a bunch of thread launch at once.
          Hide
          Henning Petersen added a comment - - edited

          I have the same symptoms using 5.0.14 on a JBoss 4.2.2 and Java 1.6.0_07/Windows XP, but a different call stack.

          I can reproduce the deadlock manually - I just need to double-/triple-/quadruple-click on a page link without the page class having been initialized before. When I warm up the app first, everything is fine and stable.

          This is the stack of the blocked threads:

          Daemon Thread [http-127.0.0.1-8080-4] (Suspended)
          Unsafe.park(boolean, long) line: not available [native method]
          LockSupport.park(Object) line: 158
          ReentrantReadWriteLock$NonfairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt() line: 712
          ReentrantReadWriteLock$NonfairSync(AbstractQueuedSynchronizer).doAcquireShared(int) line: 842
          ReentrantReadWriteLock$NonfairSync(AbstractQueuedSynchronizer).acquireShared(int) line: 1162
          ReentrantReadWriteLock$ReadLock.lock() line: 594
          ConcurrentBarrier.restoreReadLock(boolean) line: 173
          ConcurrentBarrier.tryWithWrite(Runnable, long, TimeUnit) line: 243
          CheckForUpdatesFilter$2.invoke() line: 87
          CheckForUpdatesFilter$2.invoke() line: 84
          ConcurrentBarrier.withRead(Invokable<T>) line: 83
          CheckForUpdatesFilter.service(Request, Response, RequestHandler) line: 106
          $RequestHandler_11c3d2f72ad.service(Request, Response) line: not available
          $RequestHandler_11c3d2f72a4.service(Request, Response) line: not available
          TapestryModule$12.service(HttpServletRequest, HttpServletResponse) line: 933
          HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57
          AnonymousProcessingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 125
          HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52
          $HttpServletRequestFilter_11c3d2f72a2.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57
          SecurityContextHolderAwareRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 81
          HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52
          $HttpServletRequestFilter_11c3d2f72a1.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57
          RememberMeProcessingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 142
          HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52
          $HttpServletRequestFilter_11c3d2f72a0.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57
          AuthenticationProcessingFilter(AbstractProcessingFilter).doFilter(ServletRequest, ServletResponse, FilterChain) line: 271
          HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52
          $HttpServletRequestFilter_11c3d2f729f.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57
          HttpSessionContextIntegrationFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 249
          HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52
          $HttpServletRequestFilter_11c3d2f729e.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          MultipartServletRequestFilter.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 44
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          IgnoredPathsFilter.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 62
          $HttpServletRequestFilter_11c3d2f72a3.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available
          $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available
          $HttpServletRequestHandler_11c3d2f729c.service(HttpServletRequest, HttpServletResponse) line: not available
          TapestrySpringFilter(TapestryFilter).doFilter(ServletRequest, ServletResponse, FilterChain) line: 177
          ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235
          ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206
          ReplyHeaderFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 96
          ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235
          ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206
          StandardWrapperValve.invoke(Request, Response) line: 230
          StandardContextValve.invoke(Request, Response) line: 175
          SecurityAssociationValve.invoke(Request, Response) line: 179
          JaccContextValve.invoke(Request, Response) line: 84
          StandardHostValve.invoke(Request, Response) line: 127
          ErrorReportValve.invoke(Request, Response) line: 102
          CachedConnectionValve.invoke(Request, Response) line: 157
          StandardEngineValve.invoke(Request, Response) line: 109
          CoyoteAdapter.service(Request, Response) line: 262
          Http11Processor.process(Socket) line: 844
          Http11Protocol$Http11ConnectionHandler.process(Socket) line: 583
          JIoEndpoint$Worker.run() line: 446
          Thread.run() line: 619

          Show
          Henning Petersen added a comment - - edited I have the same symptoms using 5.0.14 on a JBoss 4.2.2 and Java 1.6.0_07/Windows XP, but a different call stack. I can reproduce the deadlock manually - I just need to double-/triple-/quadruple-click on a page link without the page class having been initialized before. When I warm up the app first, everything is fine and stable. This is the stack of the blocked threads: Daemon Thread [http-127.0.0.1-8080-4] (Suspended) Unsafe.park(boolean, long) line: not available [native method] LockSupport.park(Object) line: 158 ReentrantReadWriteLock$NonfairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt() line: 712 ReentrantReadWriteLock$NonfairSync(AbstractQueuedSynchronizer).doAcquireShared(int) line: 842 ReentrantReadWriteLock$NonfairSync(AbstractQueuedSynchronizer).acquireShared(int) line: 1162 ReentrantReadWriteLock$ReadLock.lock() line: 594 ConcurrentBarrier.restoreReadLock(boolean) line: 173 ConcurrentBarrier.tryWithWrite(Runnable, long, TimeUnit) line: 243 CheckForUpdatesFilter$2.invoke() line: 87 CheckForUpdatesFilter$2.invoke() line: 84 ConcurrentBarrier.withRead(Invokable<T>) line: 83 CheckForUpdatesFilter.service(Request, Response, RequestHandler) line: 106 $RequestHandler_11c3d2f72ad.service(Request, Response) line: not available $RequestHandler_11c3d2f72a4.service(Request, Response) line: not available TapestryModule$12.service(HttpServletRequest, HttpServletResponse) line: 933 HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57 AnonymousProcessingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 125 HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52 $HttpServletRequestFilter_11c3d2f72a2.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57 SecurityContextHolderAwareRequestFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 81 HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52 $HttpServletRequestFilter_11c3d2f72a1.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57 RememberMeProcessingFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 142 HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52 $HttpServletRequestFilter_11c3d2f72a0.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57 AuthenticationProcessingFilter(AbstractProcessingFilter).doFilter(ServletRequest, ServletResponse, FilterChain) line: 271 HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52 $HttpServletRequestFilter_11c3d2f729f.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available HttpServletRequestFilterWrapper$1.doFilter(ServletRequest, ServletResponse) line: 57 HttpSessionContextIntegrationFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 249 HttpServletRequestFilterWrapper.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 52 $HttpServletRequestFilter_11c3d2f729e.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available MultipartServletRequestFilter.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 44 $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available IgnoredPathsFilter.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: 62 $HttpServletRequestFilter_11c3d2f72a3.service(HttpServletRequest, HttpServletResponse, HttpServletRequestHandler) line: not available $HttpServletRequestHandler_11c3d2f72a5.service(HttpServletRequest, HttpServletResponse) line: not available $HttpServletRequestHandler_11c3d2f729c.service(HttpServletRequest, HttpServletResponse) line: not available TapestrySpringFilter(TapestryFilter).doFilter(ServletRequest, ServletResponse, FilterChain) line: 177 ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235 ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206 ReplyHeaderFilter.doFilter(ServletRequest, ServletResponse, FilterChain) line: 96 ApplicationFilterChain.internalDoFilter(ServletRequest, ServletResponse) line: 235 ApplicationFilterChain.doFilter(ServletRequest, ServletResponse) line: 206 StandardWrapperValve.invoke(Request, Response) line: 230 StandardContextValve.invoke(Request, Response) line: 175 SecurityAssociationValve.invoke(Request, Response) line: 179 JaccContextValve.invoke(Request, Response) line: 84 StandardHostValve.invoke(Request, Response) line: 127 ErrorReportValve.invoke(Request, Response) line: 102 CachedConnectionValve.invoke(Request, Response) line: 157 StandardEngineValve.invoke(Request, Response) line: 109 CoyoteAdapter.service(Request, Response) line: 262 Http11Processor.process(Socket) line: 844 Http11Protocol$Http11ConnectionHandler.process(Socket) line: 583 JIoEndpoint$Worker.run() line: 446 Thread.run() line: 619
          Hide
          Howard M. Lewis Ship added a comment -

          Man this is tricky.

          Java loads classes as needed. Accessing a method, such as PublicList.getQueryPublicCallback(), is referencing a previously unloaded class, which is then loaded. The Java library code is synchronized against the class loader instance, while other code in T5 is synchronized against the global mutex.

          This is a bit like whack-a-mole.

          I think I need to be able to reproduce this on my workstation (duh!).

          I would think that synchronizing on the global mutex inside PackageAwareLoader.findClass() would help ... but that caused a prior deadlock (I commented out the code at some point).

          Looking at the traces, this kind of thing should only happen on the first load of the page (i.e., a race by multiple threads to initialize the page instances). Is this true? If you "warm up the app" then start crushing it with requests, does it fail? In other words, I could use a little more step-by-step in how you reach the deadlock, and what operating systems are involved.

          Show
          Howard M. Lewis Ship added a comment - Man this is tricky. Java loads classes as needed. Accessing a method, such as PublicList.getQueryPublicCallback(), is referencing a previously unloaded class, which is then loaded. The Java library code is synchronized against the class loader instance, while other code in T5 is synchronized against the global mutex. This is a bit like whack-a-mole. I think I need to be able to reproduce this on my workstation (duh!). I would think that synchronizing on the global mutex inside PackageAwareLoader.findClass() would help ... but that caused a prior deadlock (I commented out the code at some point). Looking at the traces, this kind of thing should only happen on the first load of the page (i.e., a race by multiple threads to initialize the page instances). Is this true? If you "warm up the app" then start crushing it with requests, does it fail? In other words, I could use a little more step-by-step in how you reach the deadlock, and what operating systems are involved.
          Hide
          Jun Tsai added a comment - - edited

          I have the same problem,when I rapidly refresh a frameset page.

          I use tapestry-core-5.0.15-20080904.110021-26

          the main dump message:

          Found one Java-level deadlock:
          =============================
          "qtp0-11":
          waiting to lock monitor 0x000000004019edd8 (object 0x00007f82278a3970, a java.lang.Object),
          which is held by "qtp0-7"
          "qtp0-7":
          waiting to lock monitor 0x00007f81f69bf728 (object 0x00007f822861cf18, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader),
          which is held by "qtp0-9"
          "qtp0-9":
          waiting to lock monitor 0x000000004019edd8 (object 0x00007f82278a3970, a java.lang.Object),
          which is held by "qtp0-7"

          "qtp0-7":
          at java.lang.ClassLoader.checkCerts(ClassLoader.java:751)

          • waiting to lock <0x00007f822861cf18> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487)
            at java.lang.ClassLoader.defineClass(ClassLoader.java:614)
            at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
            at java.lang.reflect.Method.invoke(Method.java:597)
            at javassist.ClassPool.toClass2(ClassPool.java:965)
          • locked <0x00007f81f80afaa0> (a java.lang.Class for javassist.ClassPool)
            at javassist.ClassPool.toClass(ClassPool.java:947)
            at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:95)
          • locked <0x00007f82278a3970> (a java.lang.Object)
          • locked <0x00007f822861d528> (a org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl)
            at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89)
            ...............................

          "qtp0-9":
          at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:184)

          • waiting to lock <0x00007f82278a3970> (a java.lang.Object)
            at javassist.Loader.findClass(Loader.java:340)
            at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:95)
            at javassist.Loader.loadClass(Loader.java:311)
          • locked <0x00007f81f895fd88> (a java.lang.String)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
            at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
          • locked <0x00007f822861cf18> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader)
            at com.ouriba.eweb.pages.admin.PublicList.getQueryPublicCallback(PublicList.java:47)
            at $PropertyConduit_11c3553e308.get($PropertyConduit_11c3553e308.java)
            at org.apache.tapestry5.internal.bindings.PropBinding.get(PropBinding.java:53)
            at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.readParameter(InternalComponentResourcesImpl.java:254)
            at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.readParameter(InternalComponentResourcesImpl.java:285)
            ......................
          Show
          Jun Tsai added a comment - - edited I have the same problem,when I rapidly refresh a frameset page. I use tapestry-core-5.0.15-20080904.110021-26 the main dump message: Found one Java-level deadlock: ============================= "qtp0-11": waiting to lock monitor 0x000000004019edd8 (object 0x00007f82278a3970, a java.lang.Object), which is held by "qtp0-7" "qtp0-7": waiting to lock monitor 0x00007f81f69bf728 (object 0x00007f822861cf18, a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader), which is held by "qtp0-9" "qtp0-9": waiting to lock monitor 0x000000004019edd8 (object 0x00007f82278a3970, a java.lang.Object), which is held by "qtp0-7" "qtp0-7": at java.lang.ClassLoader.checkCerts(ClassLoader.java:751) waiting to lock <0x00007f822861cf18> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at java.lang.ClassLoader.preDefineClass(ClassLoader.java:487) at java.lang.ClassLoader.defineClass(ClassLoader.java:614) at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at javassist.ClassPool.toClass2(ClassPool.java:965) locked <0x00007f81f80afaa0> (a java.lang.Class for javassist.ClassPool) at javassist.ClassPool.toClass(ClassPool.java:947) at org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl.createClass(CtClassSourceImpl.java:95) locked <0x00007f82278a3970> (a java.lang.Object) locked <0x00007f822861d528> (a org.apache.tapestry5.ioc.internal.services.CtClassSourceImpl) at org.apache.tapestry5.ioc.internal.services.AbstractFab.createClass(AbstractFab.java:89) ............................... "qtp0-9": at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl.onLoad(ComponentInstantiatorSourceImpl.java:184) waiting to lock <0x00007f82278a3970> (a java.lang.Object) at javassist.Loader.findClass(Loader.java:340) at org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader.findClass(ComponentInstantiatorSourceImpl.java:95) at javassist.Loader.loadClass(Loader.java:311) locked <0x00007f81f895fd88> (a java.lang.String) at java.lang.ClassLoader.loadClass(ClassLoader.java:251) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) locked <0x00007f822861cf18> (a org.apache.tapestry5.internal.services.ComponentInstantiatorSourceImpl$PackageAwareLoader) at com.ouriba.eweb.pages.admin.PublicList.getQueryPublicCallback(PublicList.java:47) at $PropertyConduit_11c3553e308.get($PropertyConduit_11c3553e308.java) at org.apache.tapestry5.internal.bindings.PropBinding.get(PropBinding.java:53) at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.readParameter(InternalComponentResourcesImpl.java:254) at org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.readParameter(InternalComponentResourcesImpl.java:285) ......................
          Hide
          Howard M. Lewis Ship added a comment -

          I've not seen any deadlocks lately, but they can be difficult to track down. I've asked the original posters to try to reproduce the deadlock, if they can't with a current 5.0.15-SNAPSHOT then we'll have to consider this issue closed.

          If it recurs, re-open this bug if 5.0.15 has not been released; clone this bug as a new bug if 5.0.15 has been released.

          Show
          Howard M. Lewis Ship added a comment - I've not seen any deadlocks lately, but they can be difficult to track down. I've asked the original posters to try to reproduce the deadlock, if they can't with a current 5.0.15-SNAPSHOT then we'll have to consider this issue closed. If it recurs, re-open this bug if 5.0.15 has not been released; clone this bug as a new bug if 5.0.15 has been released.
          Hide
          Francois Armand added a comment -

          I believe that OSGi solved this kind of deadlock with a "peer class loading" pattern in place of the classic hierarchical one. I'm not quite familiar with classloading issue, but I found this link that seem to be related http://www.ibm.com/developerworks/java/library/j-dclp4/index.html#main . Hope it might help...

          Show
          Francois Armand added a comment - I believe that OSGi solved this kind of deadlock with a "peer class loading" pattern in place of the classic hierarchical one. I'm not quite familiar with classloading issue, but I found this link that seem to be related http://www.ibm.com/developerworks/java/library/j-dclp4/index.html#main . Hope it might help...
          Hide
          Bruno Bonfils added a comment -

          Hi,

          I have the same problem with the deadlock using apache-tomcat (tried both 5 and 6) when I stress our application using jmeter. Most of time the lock occurs with 30 simultaneaous virtual users (with sequential requests, w/o pause), sometimes only with 10 VU :/

          I tried version 14 and 15-SNAPSHOT, got the same problem.

          Thanks

          Show
          Bruno Bonfils added a comment - Hi, I have the same problem with the deadlock using apache-tomcat (tried both 5 and 6) when I stress our application using jmeter. Most of time the lock occurs with 30 simultaneaous virtual users (with sequential requests, w/o pause), sometimes only with 10 VU :/ I tried version 14 and 15-SNAPSHOT, got the same problem. Thanks
          Hide
          Howard M. Lewis Ship added a comment -

          Seems like anything that goes near the class loader can cause this problem. I put an additional synchronized block around PropertyAccessImpl's use of the java.beans.Introspector. That fixed the hang running the tapestry-core tests. At this rate, though, I'm leaning towards a solution where a single thread is responsible for anything that might involve building classes or loading classes. That might get somewhat complicated, though.

          Show
          Howard M. Lewis Ship added a comment - Seems like anything that goes near the class loader can cause this problem. I put an additional synchronized block around PropertyAccessImpl's use of the java.beans.Introspector. That fixed the hang running the tapestry-core tests. At this rate, though, I'm leaning towards a solution where a single thread is responsible for anything that might involve building classes or loading classes. That might get somewhat complicated, though.
          Hide
          Howard M. Lewis Ship added a comment -

          I'm now getting a deadlock when running the tapestry-core test suite. Interesting.

          Show
          Howard M. Lewis Ship added a comment - I'm now getting a deadlock when running the tapestry-core test suite. Interesting.
          Hide
          Howard M. Lewis Ship added a comment -

          I've implemented the global MUTEX.

          It is used for module class instantation, service proxy creation as well as serializing loading/transformation of component classes. On an application that is heavily accessed immediately upon startup, we should see one thread largely monopolize the creation of new service proxies and component class loading while other threads block. This is neither a good thing nor a bad thing.

          Show
          Howard M. Lewis Ship added a comment - I've implemented the global MUTEX. It is used for module class instantation, service proxy creation as well as serializing loading/transformation of component classes. On an application that is heavily accessed immediately upon startup, we should see one thread largely monopolize the creation of new service proxies and component class loading while other threads block. This is neither a good thing nor a bad thing.
          Hide
          Howard M. Lewis Ship added a comment -

          From what I can tell, this occurs because more than one thread is attempting to define a class at the same time.

          In both examples that we've gotten, one thread is defining a new class on the context class loader (i.e., defining service proxies, etc.) while another is defining a new class on the component class loader (a child of the context class loader). You wouldn't think they interfere, but it appears that adding a class on the child affects the parent. I still don't get why this is a deadlock however.

          I'm beginning to think there needs to be a global mutex for this purpose

          Show
          Howard M. Lewis Ship added a comment - From what I can tell, this occurs because more than one thread is attempting to define a class at the same time. In both examples that we've gotten, one thread is defining a new class on the context class loader (i.e., defining service proxies, etc.) while another is defining a new class on the component class loader (a child of the context class loader). You wouldn't think they interfere, but it appears that adding a class on the child affects the parent. I still don't get why this is a deadlock however. I'm beginning to think there needs to be a global mutex for this purpose
          Hide
          Howard M. Lewis Ship added a comment -

          Looks like 2468 isn't dead yet.

          Show
          Howard M. Lewis Ship added a comment - Looks like 2468 isn't dead yet.
          Hide
          Steven Woolley added a comment -

          Full dump (thank you kill -QUIT!)

          Show
          Steven Woolley added a comment - Full dump (thank you kill -QUIT!)

            People

            • Assignee:
              Howard M. Lewis Ship
              Reporter:
              Steven Woolley
            • Votes:
              7 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development