Tapestry 5
  1. Tapestry 5
  2. TAP5-1929

High contention in method InternalComponentResourcesImpl.postRenderCleanup() and NamedSet.getValues()

    Details

      Description

      From the mailing list:

      we want to rollout a Tapestry app very shortly, but we struggle with
      load testing issues. We are currently load testing on one Tomcat 6.0.32:

      • 500 worker threads, tapestry.production-mode=true
      • Intel(R) Xeon(R) CPU X7460 @ 2.66GHz
      • OpenJDK Runtime Environment (IcedTea6 1.7.10)
        (rhel-1.20.b17.el5-x86_64) OpenJDK 64-Bit Server VM (build 14.0-b16,
        mixed mode))
        and 2 loadrunner test clients.

      After ramping up the concurrent requests (about 5min) we reach the
      maximum at about 450req/sec and get server busy errors. We see a high
      thread contention on InternalComponentResourcesImpl.postRenderCleanup
      currently with the Loop component, as there 10 Loop on the Index page.
      Is there a workaround possible without removing the Loop component from
      the page to increase the throughput? The thread dumps series looks like
      this: 1 thread locks 0x00000000e3858990 and over 400 threads are
      waiting. This lock is persistent over a thread dumps series. I guess the
      private synchronized Map<String, Object> getRenderVariables(boolean
      create) call hits us.

      "http-9080-188" daemon prio=10 tid=0x000000004d463000 nid=0x382a
      runnable [0x0000000055b2f000]
      java.lang.Thread.State: RUNNABLE
      at
      org.apache.tapestry5.internal.transform.ParameterWorker$3$1.getState(ParameterWorker.java:206)
      at
      org.apache.tapestry5.internal.transform.ParameterWorker$3$1.reset(ParameterWorker.java:302)
      at
      org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl$1.work(InternalComponentResourcesImpl.java:136)
      at
      org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl$1.work(InternalComponentResourcesImpl.java:133)
      at
      org.apache.tapestry5.internal.util.NamedSet.eachValue(NamedSet.java:171)

      • locked <0x00000000e3858990> (a
        org.apache.tapestry5.internal.util.NamedSet)
        at
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.resetParameterConduits(InternalComponentResourcesImpl.java:546)
      • locked <0x00000000e385c038> (a
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl)
        at
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.postRenderCleanup(InternalComponentResourcesImpl.java:522)
        at
        org.apache.tapestry5.corelib.components.Loop.postRenderCleanup(Loop.java)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl$1.run(ComponentPageElementImpl.java:85)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl.invoke(ComponentPageElementImpl.java:956)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl.access$1800(ComponentPageElementImpl.java:61)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl$PostRenderCleanupPhase.render(ComponentPageElementImpl.java:443)
        at
        org.apache.tapestry5.internal.services.RenderQueueImpl.run(RenderQueueImpl.java:72)
        at
        org.apache.tapestry5.internal.services.PageRenderQueueImpl.render(PageRenderQueueImpl.java:124)
        at $PageRenderQueue_135675e1f6b934.render(Unknown Source)
        at $PageRenderQueue_135675e1f6b933.render(Unknown Source)
        at
        org.apache.tapestry5.internal.services.MarkupRendererTerminator.renderMarkup(MarkupRendererTerminator.java:37)
        ...
        "http-9080-499" daemon prio=10 tid=0x000000004dffb000 nid=0x3b7d waiting
        for monitor entry [0x0000000069063000]
        java.lang.Thread.State: BLOCKED (on object monitor)
        at
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.getRenderVariables(InternalComponentResourcesImpl.java:476)
      • waiting to lock <0x00000000e385c038> (a
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl)
        at
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.postRenderCleanup(InternalComponentResourcesImpl.java:517)
        at
        org.apache.tapestry5.corelib.components.Loop.postRenderCleanup(Loop.java)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl$1.run(ComponentPageElementImpl.java:85)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl.invoke(ComponentPageElementImpl.java:956)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl.access$1800(ComponentPageElementImpl.java:61)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl$PostRenderCleanupPhase.render(ComponentPageElementImpl.java:443)
        at
        org.apache.tapestry5.internal.services.RenderQueueImpl.run(RenderQueueImpl.java:72)
        at
        org.apache.tapestry5.internal.services.PageRenderQueueImpl.render(PageRenderQueueImpl.java:124)
        at $PageRenderQueue_135675e1f6b934.render(Unknown Source)
        at $PageRenderQueue_135675e1f6b933.render(Unknown Source)
        at
        org.apache.tapestry5.internal.services.MarkupRendererTerminator.renderMarkup(MarkupRendererTerminator.java:37)
        ...
        Additionally we experienced a similar issue when using a component with
        a mixin annotation in a loop, that was rendered more than 20 times on
        the page.
        The contention here was on the
        org.apache.tapestry5.internal.util.NamedSet.getValues call

      "http-9080-79" daemon prio=10 tid=0x00000000588dc000 nid=0x3e61 waiting
      for monitor entry [0x000000004ad98000]
      java.lang.Thread.State: BLOCKED (on object monitor)
      at
      org.apache.tapestry5.internal.util.NamedSet.getValues(NamedSet.java:78)

      • waiting to lock <0x00000000e2fa4d70> (a
        org.apache.tapestry5.internal.util.NamedSet)
        at
        org.apache.tapestry5.internal.util.NamedSet.getValues(NamedSet.java:257)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl.mixinForClassName(ComponentPageElementImpl.java:892)
        at
        org.apache.tapestry5.internal.structure.ComponentPageElementImpl.getMixinByClassName(ComponentPageElementImpl.java:879)mvn
        at
        org.apache.tapestry5.internal.structure.InternalComponentResourcesImpl.getMixinByClassName(InternalComponentResourcesImpl.java:366)
        at
        org.apache.tapestry5.internal.transform.MixinWorker$1$1.get(MixinWorker.java:92)
        at
        biz.toc.buyme.client.webapp.core.components.DHTMLTimer.conduit_get__informalElement(DHTMLTimer.java)

      Since these are areas of high contention, we can change them to use an explicit Lock instance so that in the normal case (all readers, no writers), there is no contention or synchronization.

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Howard M. Lewis Ship
            Reporter:
            Howard M. Lewis Ship
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development