[JCR-2623] Deadlock in cluster when registering node types

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.5, 2.0
• Component/s: clustering
• Assignee: Unassigned
• Reporter: Thomas Draier
• Created: 06/May/10 09:50
• Updated: 27/Sep/11 21:39
• Resolved: 28/Jan/11 06:24

Description

Hi,

I've got a deadlock when using clusters and making node types registration on both nodes at the same time - One of the node starts reading journal entries before registering the node type, and is then blocked when trying to register types - both thread locks the journal and the node types registry :

"ClusterNode-exp01" id=151 idx=0x244 tid=2384 prio=5 alive, in native, blocked, daemon
– Blocked trying to get lock: org/apache/jackrabbit/core/nodetype/NodeTypeRegistry@0x00000001923C9660[thin lock]
at jrockit/vm/Threads.sleep(I)V(Native Method)
at jrockit/vm/Locks.waitForThinRelease(Locks.java:1209)
at jrockit/vm/Locks.monitorEnterSecondStageHard(Locks.java:1342)
at jrockit/vm/Locks.monitorEnterSecondStage(Locks.java:1259)
at jrockit/vm/Locks.monitorEnter(Locks.java:2466)
at org/apache/jackrabbit/core/nodetype/NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:223)
at org/apache/jackrabbit/core/nodetype/NodeTypeRegistry.externalRegistered(NodeTypeRegistry.java:691)
at org/apache/jackrabbit/core/cluster/ClusterNode.process(ClusterNode.java:925)
at org/apache/jackrabbit/core/cluster/NodeTypeRecord.process(NodeTypeRecord.java:202)
at org/apache/jackrabbit/core/cluster/ClusterNode.consume(ClusterNode.java:798)
at org/apache/jackrabbit/core/journal/AbstractJournal.doSync(AbstractJournal.java:213)
at org/apache/jackrabbit/core/journal/AbstractJournal.sync(AbstractJournal.java:188)
at org/apache/jackrabbit/core/cluster/ClusterNode.sync(ClusterNode.java:315)
at org/apache/jackrabbit/core/cluster/ClusterNode.run(ClusterNode.java:286)
at java/lang/Thread.run(Thread.java:619)

...

"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x44 tid=4348 prio=5 alive, in native, waiting, daemon
– Waiting for notification on: EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$WriterLock@0x00000001921DD110[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)[optimized]
at EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$WriterLock.acquire(WriterPreferenceReadWriteLock.java:240)
^-- Lock released while waiting: EDU/oswego/cs/dl/util/concurrent/WriterPreferenceReadWriteLock$WriterLock@0x00000001921DD110[fat lock]
at org/apache/jackrabbit/core/journal/AbstractJournal.lockAndSync(AbstractJournal.java:250)
at org/apache/jackrabbit/core/journal/DefaultRecordProducer.append(DefaultRecordProducer.java:51)
at org/apache/jackrabbit/core/cluster/ClusterNode.registered(ClusterNode.java:468)
at org/apache/jackrabbit/core/nodetype/NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:233)
^-- Holding lock: org/apache/jackrabbit/core/nodetype/NodeTypeRegistry@0x00000001923C9660[thin lock]
at org/apache/jackrabbit/core/nodetype/NodeTypeRegistry.registerNodeTypes(NodeTypeRegistry.java:208)
at org/apache/jackrabbit/core/nodetype/NodeTypeManagerImpl.registerNodeTypes(NodeTypeManagerImpl.java:431)
at org/apache/jackrabbit/core/nodetype/NodeTypeManagerImpl.registerNodeTypes(NodeTypeManagerImpl.java:282)

• thread1 reads an entry from the journal, acquire a read lock on the Journal.rwLock, start reading entries.

• Second thread start registering new node types. It gets the NodeTypeRegistry lock. Then tries to write journal entries for new node types - but blocks on getting writeLock for Journal.rwLock.

• Back to first thread, continues reading journal entries, find a NodeTypeRecord of type register - tries to register the nodeType. NodeTypeRegistry is still own by thread2, both threads are blocked.

Issue Links

• duplicates: JCR-2866 Cluster: Node type register/unregister deadlock