[ARTEMIS-3509] STOMP Websocket client disconnected after ConcurrentModificationException in Broker

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.16.0
• Component/s: Broker
• Assignee: Unassigned
• Reporter: Elodie Vandromme
• Created: 01/Oct/21 07:11
• Updated: 03/Mar/22 18:20

Description

Web clients connected through WebSocket are randomly disconnected from artemis broker without notification of the disconnection.

A ConcurrentModificationException is raised;

[2021-09-28 10:26:53,591] - [WILDFLY] - [WARN ] - [Thread-11 ([ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba|mailto:ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@444dc7ba])] - [org.apache.activemq.artemis.core.server] : *AMQ222151: removing consumer which did not handle a message*, consumer=ServerConsumerImpl [id=1990, filter=null, binding=LocalQueueBinding [address=jms.topic.defended_assetFull, queue=QueueImpl[name=d1586796-2035-11ec-8d2e-00505689a76b, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=2f00c36f-2034-11ec-8d2e-00505689a76b], temp=true]@4180826b, filter=null, name=d1586796-2035-11ec-8d2e-00505689a76b, clusterName=d1586796-2035-11ec-8d2e-00505689a76b2f00c36f-2034-11ec-8d2e-00505689a76b]], message=Reference[2019]:RELIABLE:CoreMessage[messageID=2019,durable=true,userID=d5e2d1c3-2035-11ec-8d2e-00505689a76b,priority=4, timestamp=Tue Sep 28 10:26:53 CEST 2021,expiration=0, durable=true, address=jms.topic.defended_assetFull,size=6520,properties=TypedProperties[__AMQ_CID=f9eab79a-2034-11ec-8d2e-00505689a76b,_AMQ_ROUTING_TYPE=0]]@155890432: java.util.ConcurrentModificationException
                    at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:909)
                    at java.util.ArrayList$Itr.next(ArrayList.java:859)
                    at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:179)
                    at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:579)
                    at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:40)
                    at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:322)
                    at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:190)
                    at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.executeOnCompletion(OperationContextImpl.java:128)
                    at org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.afterCompleteOperations(AbstractJournalStorageManager.java:321)
                    at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:313)
                    at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.commit(TransactionImpl.java:264)
                    at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.commit(ServerSessionImpl.java:1284)
                    at org.apache.activemq.artemis.core.protocol.stomp.StompSession$1.run(StompSession.java:164)
                    at org.apache.activemq.artemis.core.protocol.stomp.StompSession.afterDelivery(StompSession.java:116)
                    at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:511)
                    at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:3744)
                    at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3048)
                    at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2400(QueueImpl.java:125)
                    at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4119)
                    at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42)
                    at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
                    at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65)
                    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
                    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
                    at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)

Artemis (2.16.0) is deployed in Wildfly (24.0.1.Final)

Clients are connected to stomp acceptor:

<acceptor name="stomp-over-ws" factory-class="org.apache.activemq.artemis.core.remoting.impl.netty.NettyAcceptorFactory">
   <param name="host" value="0.0.0.0" />
   <param name="protocols" value="STOMP" />
   <param name="port" value="61614" />
</acceptor>

Bug is random but is reproduced quite often with a full scenario we have (on one topic or another) so we are able to reproduce it and add logs if necessary.

Issue Links

• links to: GitHub Pull Request #3952