When running Groovy without -Dgroovy.use.classvalue=true the ClassInfo instances are cached in a ManagedConcurrentMap (MCM). New values are computed on demand and computation involves both a lock on a segment within the MCM and a lock on the GlobalClassSet (GCS) which is backed by a ManagedLinkedList. The problem is that both the ManagedConcurrentMap and the GlobalClassSet share the same ReferenceQueue.
Assume there is an enqueued ClassInfo value that is stored in Segment2 of the MCM. Now assume that Thread1 and Thread2 both request ClassInfo.getClassInfo(..) for two different classes that do not currently exist in the cache. Assume that based on hashing Thread1 gets a lock on Segment1 and Thread2 gets a lock on Segment2. Assume that Thread1 is the first to call computeValue which in turn calls GlobalClassSet.add(..). This call adds a new value to a ManagedLinkedList, and since it's managed the add operation will process the ReferenceQueue. So Thread1 will attempt to dequeue the ClassInfo and the finalizeReference method on it's entry will attempt to remove it from Segment2. Thread2 holds the lock for Segment2 and Thread2 is blocked and can't progress it's waiting on the the lock Thread1 holds the lock for the GlobalClassSet, so deadlock occurs.
The attached test case includes a thread dump at the bottom.