Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.6.1, 2.0
    • Fix Version/s: 1.6.2, 2.1
    • Component/s: jackrabbit-core
    • Labels:
    • Environment:
      Weblogic 9.2

      Description

      In one of our client deployments on WebLogic 9.2 we observed JackRabbit sessions going stale in a load test. This was observed against release 1.6.1 (to which we migrated due to concurrency related issues JCR-2081 and JCR-2237). Same effect with 2.0.0.

      I could finally reproduce this issue locally. And it seems to boil down to WLS invoking the sequence of <prepare> ... <release> ... <commit> on one XA session from multiple threads, as it seems breaking assumptions of the thread-bound java.util.concurrent-RWLock based DefaultISMLocking class.
      Effectively the setActiveXid(..) method on DefaultISMLocking$RWLock fails as the old active XID was not yet cleared. With the result of more and more sessions deadlocking in below's invocation stack.

      "[ACTIVE] ExecuteThread: '27' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=1 tid=0x33fc3ec0 nid=0x2324 in Object.wait() [0x2156a000..0x2156beb0] at java.lang.Object.wait(Native Method) - waiting on <0x68a54698> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock) at java.lang.Object.wait(Object.java:474) at EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock.acquire(Unknown Source) - locked <0x68a54698> (a EDU.oswego.cs.dl.util.concurrent.WriterPreferenceReadWriteLock$WriterLock) at org.apache.jackrabbit.core.state.DefaultISMLocking$1.<init>(DefaultISMLocking.java:64) at org.apache.jackrabbit.core.state.DefaultISMLocking.acquireWriteLock(DefaultISMLocking.java:61) at org.apache.jackrabbit.core.version.AbstractVersionManager.acquireWriteLock(AbstractVersionManager.java:146) at org.apache.jackrabbit.core.version.XAVersionManager$1.prepare(XAVersionManager.java:562) at org.apache.jackrabbit.core.TransactionContext.prepare(TransactionContext.java:154) - locked <0x6dc2ad88> (a org.apache.jackrabbit.core.TransactionContext) at org.apache.jackrabbit.core.XASessionImpl.prepare(XASessionImpl.java:331) at org.apache.jackrabbit.jca.TransactionBoundXAResource.prepare(TransactionBoundXAResource.java:68) at weblogic.connector.security.layer.AdapterLayer.prepare(AdapterLayer.java:397) at weblogic.connector.transaction.outbound.XAWrapper.prepare(XAWrapper.java:297) at weblogic.transaction.internal.XAServerResourceInfo.prepare(XAServerResourceInfo.java:1276) at weblogic.transaction.internal.XAServerResourceInfo.prepare(XAServerResourceInfo.java:499) at weblogic.transaction.internal.ServerSCInfo$1.execute(ServerSCInfo.java:335) at weblogic.kernel.Kernel.executeIfIdle(Kernel.java:243) at weblogic.transaction.internal.ServerSCInfo.startPrepare(ServerSCInfo.java:326) at weblogic.transaction.internal.ServerTransactionImpl.localPrepare(ServerTransactionImpl.java:2516) at weblogic.transaction.internal.ServerTransactionImpl.globalPrepare(ServerTransactionImpl.java:2211) at weblogic.transaction.internal.ServerTransactionImpl.internalCommit(ServerTransactionImpl.java:266) at weblogic.transaction.internal.ServerTransactionImpl.commit(ServerTransactionImpl.java:227) at weblogic.transaction.internal.TransactionManagerImpl.commit(TransactionManagerImpl.java:283) at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1028) at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:709) at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:678)
      
      1. LockManager.patch
        4 kB
        Claus Köll
      2. Xid_v4.patch
        12 kB
        Claus Köll
      3. Xid_v3.patch
        5 kB
        Claus Köll
      4. Xid_v2 + SynchronizedRef for markedXid.jpg
        63 kB
        Robert Sauer
      5. Xid_v2 deadlock.jpg
        62 kB
        Robert Sauer
      6. Xid_v2.patch
        5 kB
        Claus Köll
      7. Xid.patch
        3 kB
        Claus Köll
      8. ConcurrentReaders.java
        5 kB
        Robert Sauer
      9. jr-core-xid-aware-ism-locking1.patch
        16 kB
        Robert Sauer

        Activity

          People

          • Assignee:
            Claus Köll
            Reporter:
            Robert Sauer
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development