[GERONIMO-3687] classloader deadlock during server startup

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.2, 2.1
• Fix Version/s: 2.0.3, 2.1.1, 2.2
• Security Level: public (Regular issues)
• Assignee: Kevan Lee Miller
• Reporter: Kevan Lee Miller
• Created: 07/Dec/07 19:22
• Updated: 12/Nov/08 02:57
• Resolved: 03/Apr/08 03:26

Description

I've been seeing regular deadlocks running Geronimo on Leopard.

It's the same basic scenario as found in GERONIMO-3141. For some reason, the work around in 3141 doesn't work for me on Leopard.

Just another hack and I'm sure we can fix this...

Here's background info...

$ ./geronimo.sh run
Using GERONIMO_BASE: /Users/kevan/geronimo-jetty6-jee5-2.0.2
Using GERONIMO_HOME: /Users/kevan/geronimo-jetty6-jee5-2.0.2
Using GERONIMO_TMPDIR: var/temp
Using JRE_HOME: /System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home

You can then send the java process a QUIT signal (kill -3 <pid>) to have java dump out the thread stack traces. The deadlock occurs during a load of an Iterator class (IIRC). The JVM is not well behaved, IMO. Here's the thread stack traces that I get:

Full thread dump Java HotSpot(TM) Client VM (1.5.0_13-119 mixed mode):

"Low Memory Detector" daemon prio=5 tid=0x01009d60 nid=0x858800 runnable [0x00000000..0x00000000]

"CompilerThread0" daemon prio=9 tid=0x01009330 nid=0x857a00 waiting on condition [0x00000000..0x00000000]

"Signal Dispatcher" daemon prio=9 tid=0x01008e60 nid=0x855e00 waiting on condition [0x00000000..0x00000000]

"Finalizer" daemon prio=8 tid=0x01007d10 nid=0x81ba00 waiting for monitor entry [0xb0a05000..0xb0a05d90]
at org.apache.geronimo.transformer.TransformerCollection.transform(TransformerCollection.java:35)
at sun.instrument.TransformerManager.transform(TransformerManager.java:122)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:155)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:82)
at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)

"Reference Handler" daemon prio=10 tid=0x01007910 nid=0x81a200 in Object.wait() [0xb0984000..0xb0984d90]
at java.lang.Object.wait(Native Method)

• waiting on <0x05a735f8> (a java.lang.ref.Reference$Lock)
  at java.lang.Object.wait(Object.java:474)
  at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
• locked <0x05a735f8> (a java.lang.ref.Reference$Lock)

"main" prio=5 tid=0x010018b0 nid=0xb0801000 waiting for monitor entry [0xb07ff000..0xb0800188]
at java.lang.ClassLoader.findBootstrapClass(Native Method)
at java.lang.ClassLoader.findBootstrapClass0(ClassLoader.java:946)
at java.lang.ClassLoader.loadClass(ClassLoader.java:308)

• locked <0x05a75b78> (a sun.misc.Launcher$ExtClassLoader)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
• locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
• locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
  at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374)
• locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader)

"VM Thread" prio=9 tid=0x01007060 nid=0x809800 runnable

"VM Periodic Task Thread" prio=9 tid=0x0100aa00 nid=0x859c00 waiting on condition

"Exception Catcher Thread" prio=10 tid=0x01001b00 nid=0x80ae00 runnable

Found one Java-level deadlock:
=============================
"Finalizer":
waiting to lock monitor 0x0081b070 (object 0x05a73660, a sun.misc.Launcher$AppClassLoader),
which is held by "main"
"main":
waiting to lock monitor 0x0081b094 (object 0x09584b40, a [[I),
which is held by "Finalizer"

Java stack information for the threads listed above:
===================================================
"Finalizer":
at org.apache.geronimo.transformer.TransformerCollection.transform(TransformerCollection.java:35)
at sun.instrument.TransformerManager.transform(TransformerManager.java:122)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java:155)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:82)
at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
"main":
at java.lang.ClassLoader.findBootstrapClass(Native Method)
at java.lang.ClassLoader.findBootstrapClass0(ClassLoader.java:946)
at java.lang.ClassLoader.loadClass(ClassLoader.java:308)

• locked <0x05a75b78> (a sun.misc.Launcher$ExtClassLoader)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
• locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:280)
• locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
  at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:374)
• locked <0x05a73660> (a sun.misc.Launcher$AppClassLoader)

Found 1 deadlock.