Issue Details (XML | Word | Printable)

Key: DIRMINA-138
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Blocker Blocker
Assignee: Trustin Lee
Reporter: Vinod Panicker
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
MINA

Deadlock in SSLFilter

Created: 06/Dec/05 04:19 PM   Updated: 03/Jan/06 04:03 PM
Return to search
Component/s: None
Affects Version/s: None
Fix Version/s: 0.9.0

Time Tracking:
Not Specified

Environment: Java 5, Linux FC3

Resolution Date: 07/Dec/05 12:08 PM


 Description  « Hide
Found a deadlock in the SSLFilter implementation. This has happened quite a few times, even with the latest 0.9 snapshots.

This deadlock is causing *all* writes to block, effectively bringing the server to a standstill.

Stack trace of IoThreadPool-1 -

Name: IoThreadPool-1
State: BLOCKED on java.lang.Object@ccbab7 owned by: Thread-20
Total blocked: 6 Total waited: 27

Stack trace:
org.apache.mina.common.support.BaseIoSession.close(BaseIoSession.java:109)
com.geodesiconline.mims.io.MIMSIoSession.close(MIMSIoSession.java:56)
com.geodesiconline.mims.io.LocalIoSession.close(LocalIoSession.java:190)
com.geodesiconline.mims.io.IoHandlerImpl.sessionClosed(IoHandlerImpl.java:102)
org.apache.mina.common.support.AbstractIoFilterChain$2.sessionClosed(AbstractIoFilterChain.java:163)
org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:453)
org.apache.mina.common.support.AbstractIoFilterChain.access$700(AbstractIoFilterChain.java:52)
org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:742)
org.apache.mina.filter.SSLFilter.sessionClosed(SSLFilter.java:376)
org.apache.mina.common.support.AbstractIoFilterChain.callNextSessionClosed(AbstractIoFilterChain.java:453)
org.apache.mina.common.support.AbstractIoFilterChain.access$700(AbstractIoFilterChain.java:52)
org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.sessionClosed(AbstractIoFilterChain.java:742)
org.apache.mina.filter.ThreadPoolFilter.processEvent(ThreadPoolFilter.java:687)
org.apache.mina.filter.ThreadPoolFilter$Worker.processEvents(ThreadPoolFilter.java:421)
org.apache.mina.filter.ThreadPoolFilter$Worker.run(ThreadPoolFilter.java:376)


Stack trace of Thread-20 -

Name: Thread-20
State: BLOCKED on org.apache.mina.transport.socket.nio.support.SocketSessionImpl@bebb88 owned by: IoThreadPool-1
Total blocked: 5 Total waited: 5

Stack trace:
org.apache.mina.filter.SSLFilter.filterWrite(SSLFilter.java:493)
org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite(AbstractIoFilterChain.java:556)
org.apache.mina.common.support.AbstractIoFilterChain.access$1200(AbstractIoFilterChain.java:52)
org.apache.mina.common.support.AbstractIoFilterChain$EntryImpl$1.filterWrite(AbstractIoFilterChain.java:773)
org.apache.mina.common.support.AbstractIoFilterChain$2.filterWrite(AbstractIoFilterChain.java:213)
org.apache.mina.common.support.AbstractIoFilterChain.callPreviousFilterWrite(AbstractIoFilterChain.java:556)
org.apache.mina.common.support.AbstractIoFilterChain.filterWrite(AbstractIoFilterChain.java:547)
org.apache.mina.transport.socket.nio.support.SocketSessionImpl.write0(SocketSessionImpl.java:131)
org.apache.mina.common.support.BaseIoSession.write(BaseIoSession.java:143)
com.geodesiconline.mims.io.MIMSIoSession.write(MIMSIoSession.java:127)
com.geodesiconline.mims.io.LocalIoSession.write(LocalIoSession.java:133)
com.geodesiconline.mims.protocol.xmpp.handler.AbstractHandler.writeStreamError(AbstractHandler.java:306)
com.geodesiconline.mims.protocol.xmpp.handler.AbstractHandler.retrieveAndSetActiveDefaultListItems(AbstractHandler.java:1522)
com.geodesiconline.mims.protocol.xmpp.handler.AbstractHandler.isStanzaBlocked(AbstractHandler.java:828)
com.geodesiconline.mims.protocol.xmpp.handler.AsynchronousStanzaHandler.writePresence(AsynchronousStanzaHandler.java:432)
com.geodesiconline.mims.protocol.xmpp.handler.CommonPresence.sendPresence(CommonPresence.java:170)
com.geodesiconline.mims.protocol.xmpp.handler.PresenceBroadcastHandler.broadcastPresenceContact(PresenceBroadcastHandler.java:237)
com.geodesiconline.mims.protocol.xmpp.handler.PresenceBroadcastHandler.processOutboundStanza(PresenceBroadcastHandler.java:71)
com.geodesiconline.mims.protocol.xmpp.handler.AsynchronousStanzaHandler.processStanza(AsynchronousStanzaHandler.java:320)
com.geodesiconline.mims.protocol.xmpp.handler.InitialPresenceHandler.sendInitialPresence(InitialPresenceHandler.java:140)
com.geodesiconline.mims.protocol.xmpp.handler.InitialPresenceHandler.processOutboundStanza(InitialPresenceHandler.java:89)
com.geodesiconline.mims.protocol.xmpp.handler.AsynchronousStanzaHandler.processStanza(AsynchronousStanzaHandler.java:320)
com.geodesiconline.mims.protocol.xmpp.handler.PresenceHandler.processStanza(PresenceHandler.java:225)
com.geodesiconline.mims.protocol.xmpp.handler.AbstractHandler.handleStanza(AbstractHandler.java:195)
com.geodesiconline.mims.protocol.xmpp.io.XMPPIoHandler.processData(XMPPIoHandler.java:464)
com.geodesiconline.mims.protocol.xmpp.io.XMPPIoHandler.messageReceived(XMPPIoHandler.java:180)
com.geodesiconline.mims.protocol.xmpp.io.XMPPIoHandler.notified(XMPPIoHandler.java:660)
com.geodesiconline.mims.event.EventManager$1.run(EventManager.java:196)
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
java.lang.Thread.run(Thread.java:595)


All the other threads are queued up behind Thread-20

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Repository Revision Date User Message
ASF #354376 Tue Dec 06 08:57:23 UTC 2005 trustin Found a few deadlocks after resolving DIRMINA-138 - Deadlock in SSLFilter, and fixed them
Files Changed
MODIFY /directory/network/trunk/src/test/org/apache/mina/common/IoFilterChainTest.java
MODIFY /directory/network/trunk/src/test/org/apache/mina/filter/codec/CumulativeProtocolDecoderTest.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/support/BaseIoSession.java
MODIFY /directory/network/trunk/src/test/org/apache/mina/filter/ThreadPoolFilterRegressionTest.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/support/SSLHandler.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/codec/ProtocolCodecFilter.java

Repository Revision Date User Message
ASF #354382 Tue Dec 06 09:11:05 UTC 2005 trustin My previous fix on DIRMINA-138 (Deadlock in SSLFilter) caused a lot of problems.
I'm reverting back and applying a brand new fix.
Files Changed
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/IoFuture.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/WriteFuture.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/SSLFilter.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/support/BaseIoSession.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/ConnectFuture.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/support/SSLHandler.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/CloseFuture.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/codec/ProtocolCodecFilter.java

Repository Revision Date User Message
ASF #354385 Tue Dec 06 09:13:41 UTC 2005 trustin Fixed another deadlock specified in DIRMINA-138 (Deadlock in SSLFilter)
Files Changed
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/SSLFilter.java

[ Permlink | « Hide ]
Trustin Lee added a comment - 06/Dec/05 04:38 PM
Could you show us the thread dump for 0.8.1? I guess 0.8 doesn't get affected by this issue.

[ Permlink | « Hide ]
Vinod Panicker added a comment - 06/Dec/05 04:52 PM
Sorry... should have been 0.9.0-SNAPSHOT

Trustin Lee made changes - 06/Dec/05 05:00 PM
Field Original Value New Value
Assignee Trustin Lee [ trustin ]
Affects Version/s 0.8.1 [ 12310370 ]
[ Permlink | « Hide ]
Trustin Lee added a comment - 06/Dec/05 05:00 PM
It was because of ioLock introduced recently. I changed BaseIoSession use itself as a lock to resolve this issue.

Trustin Lee made changes - 06/Dec/05 05:00 PM
Fix Version/s 0.9 [ 11069 ]
Resolution Fixed [ 1 ]
Status Open [ 1 ] Resolved [ 5 ]
[ Permlink | « Hide ]
Trustin Lee added a comment - 07/Dec/05 12:25 AM
I need to investigate on this issue a little bit more.

Trustin Lee made changes - 07/Dec/05 12:25 AM
Status Resolved [ 5 ] Reopened [ 4 ]
Resolution Fixed [ 1 ]
Repository Revision Date User Message
ASF #354680 Wed Dec 07 02:56:45 UTC 2005 trustin Resolved issue: DIRMINA-130 (SSLFilter has to dispose SSLEngine when the filter is removed from a chain.)
* Renamed SSLHandler.release() to destroy() and added more cleanup code
Resolved issue: DIRMINA-138 (Deadlock in SSLFilter)
* Added SSLHandler.init() so SSLHandler instance can be reused over and over
* BaseIoSession doesn't hold lock for a long time anymore.
* SSLFilter uses SSLHandler instance to acquire a lock now. (instead of IoSession)
Files Changed
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/IoFuture.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/SSLFilter.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/IoSession.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/support/BaseIoSession.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/filter/support/SSLHandler.java
MODIFY /directory/network/trunk/src/java/org/apache/mina/common/CloseFuture.java

[ Permlink | « Hide ]
Trustin Lee added a comment - 07/Dec/05 12:08 PM
I reorganized acquiring locks. Now it works without a glitch.

Trustin Lee made changes - 07/Dec/05 12:08 PM
Resolution Fixed [ 1 ]
Status Reopened [ 4 ] Resolved [ 5 ]
[ Permlink | « Hide ]
Vinod Panicker added a comment - 03/Jan/06 04:03 PM
verified fixed.

Vinod Panicker made changes - 03/Jan/06 04:03 PM
Status Resolved [ 5 ] Closed [ 6 ]

Powered by a free Atlassian JIRA open source license for Apache Software Foundation. Try JIRA - bug tracking software for your team.
Atlassian JIRA the Professional Issue Tracker. (Enterprise Edition, Version: 3.13.2-#335) - Bug/feature request - Atlassian news - Contact Administrators