ActiveMQ
  1. ActiveMQ
  2. AMQ-4092

ConcurrentModificationException when using message groups

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.6.0
    • Fix Version/s: 5.9.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      ActiveMQ 5.6.0 standalone, kahadb persistence, Spring JMSTemplate producer (ObjectMessage), Camel 2.9.2 route consumer (AMQ connection pool, vm transport)

      Description

      seeing these WARN messages frequently while consuming (maxConcurrentConsumers=5) from a queue with the message groups header (JMSXGroupID) set. If I disable the message groups header, these errors go away...

      That said, I don't see any specific errors with my application (at this point), just these messages in my logs.

      2012-10-04 02:34:02,552 | WARN - Service - | Async error occurred: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
      java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
      at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
      at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:785)
      at org.apache.activemq.broker.region.Queue.send(Queue.java:707)
      at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
      at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:503)
      at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:305)
      at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
      at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
      at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:306)
      at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
      at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:453)
      at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:681)
      at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
      at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150)
      at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
      at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
      at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:231)
      at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
      at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:680)
      Caused by: java.util.ConcurrentModificationException
      at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
      at java.util.HashMap$KeyIterator.next(HashMap.java:828)
      at org.apache.activemq.util.MarshallingSupport.marshalPrimitiveMap(MarshallingSupport.java:64)
      at org.apache.activemq.command.Message.beforeMarshall(Message.java:210)
      at org.apache.activemq.command.ActiveMQObjectMessage.beforeMarshall(ActiveMQObjectMessage.java:199)
      at org.apache.activemq.openwire.v6.MessageMarshaller.looseMarshal(MessageMarshaller.java:277)
      at org.apache.activemq.openwire.v6.ActiveMQMessageMarshaller.looseMarshal(ActiveMQMessageMarshaller.java:111)
      at org.apache.activemq.openwire.v6.ActiveMQObjectMessageMarshaller.looseMarshal(ActiveMQObjectMessageMarshaller.java:111)
      at org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:168)
      at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.addMessage(KahaDBStore.java:429)
      at org.apache.activemq.store.kahadb.KahaDBStore$StoreQueueTask.run(KahaDBStore.java:1177)
      ... 3 more

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        348d 7h 10m 1 Gary Tully 17/Sep/13 11:06
        Hide
        Amol Verma added a comment -

        thanks Gary..i will give it a try.


        Thanks & Regards
        Amol Verma

        Show
        Amol Verma added a comment - thanks Gary..i will give it a try. – Thanks & Regards Amol Verma
        Gary Tully made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Gary Tully added a comment -

        fix in http://git-wip-us.apache.org/repos/asf/activemq/commit/dd91e859

        JMSXGroupFirstForConsumer property is now backed by a message attribute which avoids contention. requires v10 wireformat

        Show
        Gary Tully added a comment - fix in http://git-wip-us.apache.org/repos/asf/activemq/commit/dd91e859 JMSXGroupFirstForConsumer property is now backed by a message attribute which avoids contention. requires v10 wireformat
        Hide
        Gary Tully added a comment -

        @Amol - are you using composite destinations in your producers?

        Show
        Gary Tully added a comment - @Amol - are you using composite destinations in your producers?
        Gary Tully made changes -
        Assignee Gary Tully [ gtully ]
        Hide
        Amol Verma added a comment -

        Gary, thanks for replying. I used the settings..but still i am getting the
        same exception and failure for same file but at random occasions i.e. the
        same file is working fine most of time and some time it is failing. I am
        using latest release of activemq i.e.5.8.0

        Please suggest.

        Show
        Amol Verma added a comment - Gary, thanks for replying. I used the settings..but still i am getting the same exception and failure for same file but at random occasions i.e. the same file is working fine most of time and some time it is failing. I am using latest release of activemq i.e.5.8.0 Please suggest.
        Hide
        Gary Tully added a comment -

        @Amol google is always your friend. It is an option on kahadb - http://activemq.apache.org/kahadb.html

        so

        <broker brokerName="broker" ... >
           <persistenceAdapter>
             <kahaDB ... concurrentStoreAndDispatchQueues ="false"/>
           </persistenceAdapter>
           ...
         </broker>
        Show
        Gary Tully added a comment - @Amol google is always your friend. It is an option on kahadb - http://activemq.apache.org/kahadb.html so <broker brokerName= "broker" ... > <persistenceAdapter> <kahaDB ... concurrentStoreAndDispatchQueues = " false " /> </persistenceAdapter> ... </broker>
        Hide
        Amol Verma added a comment - - edited

        Hi,

        i am not an expert using Activemq..but i am facing the same issue. Gary Tully can you please suggest where to add, i.e. under which section, the fix code i.e. "concurrentStoreAndDispatchQueues=false" in activemq.xml? or please provide sample activemq.xml file with this workaround.

        Thanks,
        amol

        Show
        Amol Verma added a comment - - edited Hi, i am not an expert using Activemq..but i am facing the same issue. Gary Tully can you please suggest where to add, i.e. under which section, the fix code i.e. "concurrentStoreAndDispatchQueues=false" in activemq.xml? or please provide sample activemq.xml file with this workaround. Thanks, amol
        Hide
        Ben O'Day added a comment -

        when this ConcurrentModificationException event occurs...does an exception get thrown back to the producer or consumer of this message or is the issue just with setting the JMSXGroupFirstForConsumer property?

        Show
        Ben O'Day added a comment - when this ConcurrentModificationException event occurs...does an exception get thrown back to the producer or consumer of this message or is the issue just with setting the JMSXGroupFirstForConsumer property?
        Hide
        can_do added a comment - - edited

        I have set the message property such as msg.setGroupID("GROUPBEGIN_" + encodedName + "GROUPEND"
        + file.length()) and msg.setGroupSequence(++i);
        From message property,I see the value of JMSXGroupFirstForConsumer is true.

        Whether this extensive property JMSXGroupFirstForConsumer can be set to false and problem can be eliminated or not.

        Show
        can_do added a comment - - edited I have set the message property such as msg.setGroupID("GROUPBEGIN_" + encodedName + " GROUPEND " + file.length()) and msg.setGroupSequence(++i); From message property,I see the value of JMSXGroupFirstForConsumer is true. Whether this extensive property JMSXGroupFirstForConsumer can be set to false and problem can be eliminated or not.
        Hide
        can_do added a comment -

        I have the same error in AMQ 5.7.0.
        The detail log is as the following:
        ////////////////begin////////////
        javax.jms.JMSException: java.util.ConcurrentModificationException
        at com.rf.emq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49)
        at com.rf.emq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1377)
        at com.rf.emq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1305)
        at com.rf.emq.ActiveMQSession.send(ActiveMQSession.java:2060)
        at com.rf.emq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:277)
        at com.rf.emq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:212)
        at com.rf.emq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
        at com.rf.emq.product.message.ByteMessageSender.produceMessageInFragmentWithReturnValue(ByteMessageSender.java:166)
        at com.rf.emq.product.web.AutoSplitLargeFileAndSendTimerTask.splitLargeFile(AutoSplitLargeFileAndSendTimerTask.java:693)
        at com.rf.emq.product.web.AutoSplitLargeFileAndSendTimerTask$SplitLargFileSubTask.run(AutoSplitLargeFileAndSendTimerTask.j
        ava:879)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
        Caused by: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
        at com.rf.emq.broker.region.Queue.doMessageSend(Queue.java:795)
        at com.rf.emq.broker.region.Queue.send(Queue.java:717)
        at com.rf.emq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
        at com.rf.emq.broker.region.RegionBroker.send(RegionBroker.java:503)
        at com.rf.emq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:311)
        at com.rf.emq.broker.BrokerFilter.send(BrokerFilter.java:129)
        at com.rf.emq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
        at com.rf.emq.broker.TransactionBroker.send(TransactionBroker.java:317)
        at com.rf.emq.broker.BrokerFilter.send(BrokerFilter.java:129)
        at com.rf.emq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
        at com.rf.emq.broker.TransportConnection.processMessage(TransportConnection.java:450)
        at com.rf.emq.command.ActiveMQMessage.visit(ActiveMQMessage.java:680)
        at com.rf.emq.broker.TransportConnection.service(TransportConnection.java:294)
        at com.rf.emq.broker.TransportConnection$1.onCommand(TransportConnection.java:152)
        at com.rf.emq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
        at com.rf.emq.transport.MutexTransport.onCommand(MutexTransport.java:50)
        at com.rf.emq.transport.vm.VMTransport.iterate(VMTransport.java:241)
        at com.rf.emq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129)
        at com.rf.emq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47)
        ... 3 more
        Caused by: java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:806)
        at java.util.HashMap$KeyIterator.next(HashMap.java:841)
        at com.rf.emq.util.MarshallingSupport.marshalPrimitiveMap(MarshallingSupport.java:64)
        at com.rf.emq.command.Message.beforeMarshall(Message.java:215)
        at com.rf.emq.openwire.v6.MessageMarshaller.looseMarshal(MessageMarshaller.java:277)
        at com.rf.emq.openwire.v6.ActiveMQMessageMarshaller.looseMarshal(ActiveMQMessageMarshaller.java:111)
        at com.rf.emq.openwire.v6.ActiveMQBytesMessageMarshaller.looseMarshal(ActiveMQBytesMessageMarshaller.java:111)
        at com.rf.emq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:168)
        at com.rf.emq.store.kahadb.KahaDBStore$KahaDBMessageStore.addMessage(KahaDBStore.java:431)
        at com.rf.emq.store.kahadb.KahaDBStore$StoreQueueTask.run(KahaDBStore.java:1192)
        ... 3 more
        ////////////////end///////////////

        Show
        can_do added a comment - I have the same error in AMQ 5.7.0. The detail log is as the following: ////////////////begin//////////// javax.jms.JMSException: java.util.ConcurrentModificationException at com.rf.emq.util.JMSExceptionSupport.create(JMSExceptionSupport.java:49) at com.rf.emq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1377) at com.rf.emq.ActiveMQConnection.syncSendPacket(ActiveMQConnection.java:1305) at com.rf.emq.ActiveMQSession.send(ActiveMQSession.java:2060) at com.rf.emq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:277) at com.rf.emq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:212) at com.rf.emq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241) at com.rf.emq.product.message.ByteMessageSender.produceMessageInFragmentWithReturnValue(ByteMessageSender.java:166) at com.rf.emq.product.web.AutoSplitLargeFileAndSendTimerTask.splitLargeFile(AutoSplitLargeFileAndSendTimerTask.java:693) at com.rf.emq.product.web.AutoSplitLargeFileAndSendTimerTask$SplitLargFileSubTask.run(AutoSplitLargeFileAndSendTimerTask.j ava:879) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) at java.util.concurrent.FutureTask.get(FutureTask.java:111) at com.rf.emq.broker.region.Queue.doMessageSend(Queue.java:795) at com.rf.emq.broker.region.Queue.send(Queue.java:717) at com.rf.emq.broker.region.AbstractRegion.send(AbstractRegion.java:407) at com.rf.emq.broker.region.RegionBroker.send(RegionBroker.java:503) at com.rf.emq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:311) at com.rf.emq.broker.BrokerFilter.send(BrokerFilter.java:129) at com.rf.emq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) at com.rf.emq.broker.TransactionBroker.send(TransactionBroker.java:317) at com.rf.emq.broker.BrokerFilter.send(BrokerFilter.java:129) at com.rf.emq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135) at com.rf.emq.broker.TransportConnection.processMessage(TransportConnection.java:450) at com.rf.emq.command.ActiveMQMessage.visit(ActiveMQMessage.java:680) at com.rf.emq.broker.TransportConnection.service(TransportConnection.java:294) at com.rf.emq.broker.TransportConnection$1.onCommand(TransportConnection.java:152) at com.rf.emq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116) at com.rf.emq.transport.MutexTransport.onCommand(MutexTransport.java:50) at com.rf.emq.transport.vm.VMTransport.iterate(VMTransport.java:241) at com.rf.emq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:129) at com.rf.emq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:47) ... 3 more Caused by: java.util.ConcurrentModificationException at java.util.HashMap$HashIterator.nextEntry(HashMap.java:806) at java.util.HashMap$KeyIterator.next(HashMap.java:841) at com.rf.emq.util.MarshallingSupport.marshalPrimitiveMap(MarshallingSupport.java:64) at com.rf.emq.command.Message.beforeMarshall(Message.java:215) at com.rf.emq.openwire.v6.MessageMarshaller.looseMarshal(MessageMarshaller.java:277) at com.rf.emq.openwire.v6.ActiveMQMessageMarshaller.looseMarshal(ActiveMQMessageMarshaller.java:111) at com.rf.emq.openwire.v6.ActiveMQBytesMessageMarshaller.looseMarshal(ActiveMQBytesMessageMarshaller.java:111) at com.rf.emq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:168) at com.rf.emq.store.kahadb.KahaDBStore$KahaDBMessageStore.addMessage(KahaDBStore.java:431) at com.rf.emq.store.kahadb.KahaDBStore$StoreQueueTask.run(KahaDBStore.java:1192) ... 3 more ////////////////end///////////////
        Gary Tully made changes -
        Fix Version/s 5.9.0 [ 12323932 ]
        Hide
        Gary Tully added a comment -

        line this up for the next release

        Show
        Gary Tully added a comment - line this up for the next release
        Hide
        Ben O'Day added a comment -

        After doing some burnin/performance testing with concurrentStoreAndDispatchQueues="false"...this setting if proving problematic and is causing unacceptable throttling of our application. Is this issue addressed in a 5.7.0? if not, any plan to address it? thanks...

        Show
        Ben O'Day added a comment - After doing some burnin/performance testing with concurrentStoreAndDispatchQueues="false"...this setting if proving problematic and is causing unacceptable throttling of our application. Is this issue addressed in a 5.7.0? if not, any plan to address it? thanks...
        Hide
        Ben O'Day added a comment -

        alright, concurrentStoreAndDispatchQueues=false appears to have resolved the issue...not sure if we should close this given the work around of if there is a code change to better handle this case w/o that setting...I'll leave it open for now...thanks again

        Show
        Ben O'Day added a comment - alright, concurrentStoreAndDispatchQueues=false appears to have resolved the issue...not sure if we should close this given the work around of if there is a code change to better handle this case w/o that setting...I'll leave it open for now...thanks again
        Hide
        Ben O'Day added a comment -

        thanks Gary, testing with concurrentStoreAndDispatchQueues=false now...so far so good

        Show
        Ben O'Day added a comment - thanks Gary, testing with concurrentStoreAndDispatchQueues=false now...so far so good
        Hide
        Gary Tully added a comment -

        message groups, makes sense, the problem is in
        org.apache.activemq.broker.region.Queue#assignGroup where the JMSXGroupFirstForConsumer property is set, but the message is being stored concurrently.
        A workaround is to disable kahadb concurrentStoreAndDispatchQueues, set to false. This will ensure that the dispatch (which can set the property) and store do not occur in parallel.

        Need to check up on the need to set that property and see if we can do it after the message is persisted such that it will work with concurrent store and dispatch.

        Show
        Gary Tully added a comment - message groups, makes sense, the problem is in org.apache.activemq.broker.region.Queue#assignGroup where the JMSXGroupFirstForConsumer property is set, but the message is being stored concurrently. A workaround is to disable kahadb concurrentStoreAndDispatchQueues, set to false. This will ensure that the dispatch (which can set the property) and store do not occur in parallel. Need to check up on the need to set that property and see if we can do it after the message is persisted such that it will work with concurrent store and dispatch.
        Ben O'Day made changes -
        Field Original Value New Value
        Description seeing these WARN messages frequently using consuming (maxConcurrentConsumers=5) from a queue with the message groups header (JMSXGroupID) set. If I disable the message groups, these go away...

        That said, I don't see any specific errors with my application (at this point), just these messages in my logs.

        2012-10-04 02:34:02,552 | WARN - Service - | Async error occurred: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
        java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:785)
        at org.apache.activemq.broker.region.Queue.send(Queue.java:707)
        at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
        at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:503)
        at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:305)
        at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
        at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
        at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:306)
        at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
        at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:453)
        at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:681)
        at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
        at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150)
        at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
        at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
        at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:231)
        at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
        at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)
        Caused by: java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$KeyIterator.next(HashMap.java:828)
        at org.apache.activemq.util.MarshallingSupport.marshalPrimitiveMap(MarshallingSupport.java:64)
        at org.apache.activemq.command.Message.beforeMarshall(Message.java:210)
        at org.apache.activemq.command.ActiveMQObjectMessage.beforeMarshall(ActiveMQObjectMessage.java:199)
        at org.apache.activemq.openwire.v6.MessageMarshaller.looseMarshal(MessageMarshaller.java:277)
        at org.apache.activemq.openwire.v6.ActiveMQMessageMarshaller.looseMarshal(ActiveMQMessageMarshaller.java:111)
        at org.apache.activemq.openwire.v6.ActiveMQObjectMessageMarshaller.looseMarshal(ActiveMQObjectMessageMarshaller.java:111)
        at org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:168)
        at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.addMessage(KahaDBStore.java:429)
        at org.apache.activemq.store.kahadb.KahaDBStore$StoreQueueTask.run(KahaDBStore.java:1177)
        ... 3 more
        seeing these WARN messages frequently while consuming (maxConcurrentConsumers=5) from a queue with the message groups header (JMSXGroupID) set. If I disable the message groups header, these errors go away...

        That said, I don't see any specific errors with my application (at this point), just these messages in my logs.

        2012-10-04 02:34:02,552 | WARN - Service - | Async error occurred: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
        java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:785)
        at org.apache.activemq.broker.region.Queue.send(Queue.java:707)
        at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:407)
        at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:503)
        at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:305)
        at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:129)
        at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96)
        at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:306)
        at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:135)
        at org.apache.activemq.broker.TransportConnection.processMessage(TransportConnection.java:453)
        at org.apache.activemq.command.ActiveMQMessage.visit(ActiveMQMessage.java:681)
        at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:292)
        at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:150)
        at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
        at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
        at org.apache.activemq.transport.vm.VMTransport.iterate(VMTransport.java:231)
        at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:122)
        at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:43)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:680)
        Caused by: java.util.ConcurrentModificationException
        at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
        at java.util.HashMap$KeyIterator.next(HashMap.java:828)
        at org.apache.activemq.util.MarshallingSupport.marshalPrimitiveMap(MarshallingSupport.java:64)
        at org.apache.activemq.command.Message.beforeMarshall(Message.java:210)
        at org.apache.activemq.command.ActiveMQObjectMessage.beforeMarshall(ActiveMQObjectMessage.java:199)
        at org.apache.activemq.openwire.v6.MessageMarshaller.looseMarshal(MessageMarshaller.java:277)
        at org.apache.activemq.openwire.v6.ActiveMQMessageMarshaller.looseMarshal(ActiveMQMessageMarshaller.java:111)
        at org.apache.activemq.openwire.v6.ActiveMQObjectMessageMarshaller.looseMarshal(ActiveMQObjectMessageMarshaller.java:111)
        at org.apache.activemq.openwire.OpenWireFormat.marshal(OpenWireFormat.java:168)
        at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.addMessage(KahaDBStore.java:429)
        at org.apache.activemq.store.kahadb.KahaDBStore$StoreQueueTask.run(KahaDBStore.java:1177)
        ... 3 more
        Ben O'Day created issue -

          People

          • Assignee:
            Gary Tully
            Reporter:
            Ben O'Day
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development