[GEODE-4615] Deadlock shutting down client cache

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.5.0
• Component/s: client/server
• Labels: pull-request-available
• Assignee: Bruce J Schuchardt
• Reporter: Bruce J Schuchardt
• Created: 06/Feb/18 22:33
• Updated: 09/Apr/18 22:36
• Resolved: 09/Feb/18 22:36

Description

A test became deadlocked when a thread executing a query was requesting a PDX type ID from its server at the same time another thread was closing the cache.  I wrote a unit test to reproduce this problem.  Here is the deadlock:

 

"ConnectionManagerJUnitTest thread":
at org.apache.geode.cache.client.internal.pooling.PooledConnection.internalDestroy(PooledConnection.java:76)
- waiting to lock <0x000000076bbc3d20> (a org.apache.geode.cache.client.internal.pooling.PooledConnection)
at org.apache.geode.cache.client.internal.pooling.PooledConnection.internalClose(PooledConnection.java:101)
at org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl$ConnectionMap.close(ConnectionManagerImpl.java:1164)
- locked <0x000000076b848f30> (a org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl$ConnectionMap)
at org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest$3.run(ConnectionManagerJUnitTest.java:613)
"main":
at org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl$ConnectionMap.addConnection(ConnectionManagerImpl.java:1059)
- waiting to lock <0x000000076b848f30> (a org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl$ConnectionMap)
at org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.addConnection(ConnectionManagerImpl.java:463)
at org.apache.geode.cache.client.internal.pooling.ConnectionManagerImpl.borrowConnection(ConnectionManagerImpl.java:245)
at org.apache.geode.cache.client.internal.pooling.ConnectionManagerJUnitTest.testThatMapCloseCausesCacheClosedException(ConnectionManagerJUnitTest.java:631)
- locked <0x000000076bbc3d20> (a org.apache.geode.cache.client.internal.pooling.PooledConnection)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

The "main" thread obtains a Connection and then, with its sync lock held, obtains another Connection to simulate the PDX registration.

The background thread waits for the other thread to be ready and then closes the ConnectionMap.

Issue Links

• links to: GitHub Pull Request #1396