I ran into a deadlock caused by the fix for
AMQ-8169 when using Stomp NIOSSLTransport (but it could probably happen for other nio ssl transports)
The newly added synchronized on the serviceRead() caused a deadlock between the transport and the TransportConnection. One thread acquired a lock on the TransportConnection and was waiting on serviceRead() to acquire the NIOSSLTransport lock. Another thread had was inside serviceRead() so it acquired the NIOSSLTransport lock and was then later waiting for the TransportConnection lock.
The main issue is that processCommand(plain) ends up being protected by the lock and since there's multiple brokers/filters running we run into a deadlock (my current deadlock happened processing a ConsumerInfo command)
To fix this we simply need to narrow the lock as it's too broad. The goal here was to protect the the reading off the channel concurrently (so really the secureRead() method so we can move the lock to secureRead() and not lock the entire serviceRead() call and that should fix the deadlock problem while still solving the initial issue which was demonstrated by the StompNIOSSL failing before this fix. I will open a new Jira shortly and push a fix.