Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-6598

SQLException when moving messages to DLQ

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 5.12.0, 5.14.3
    • None
    • JDBC
    • None

    Description

      ActiveMQ raises a SQLException when moving messages from two or more queues to the configured dead-letter-queue. It only happens with two or more queues and multiple messages. The exception is logged on level WARN and it seems that no messages are lost. First affected version is 5.12.0, but also happens with the current version 5.14.3.

      java.io.IOException: Failed to broker message: ID:server-amq-46753-1486658241250-4:5:3:1:13 in container: java.sql.SQLException: org.apache.commons.dbcp2.DelegatingPreparedStatement with address: "NULL" is closed.
              at org.apache.activemq.util.IOExceptionSupport.create(IOExceptionSupport.java:34) ~[activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.store.jdbc.JDBCMessageStore.addMessage(JDBCMessageStore.java:163) ~[activemq-jdbc-store-5.14.3.jar:5.14.3]
              at org.apache.activemq.store.memory.MemoryTransactionStore.addMessage(MemoryTransactionStore.java:352) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.store.memory.MemoryTransactionStore$1.asyncAddQueueMessage(MemoryTransactionStore.java:159) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.Queue.doMessageSend(Queue.java:837) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.Queue.send(Queue.java:727) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.AbstractRegion.send(AbstractRegion.java:503) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.RegionBroker.send(RegionBroker.java:468) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.jmx.ManagedRegionBroker.send(ManagedRegionBroker.java:293) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.BrokerFilter.send(BrokerFilter.java:153) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.scheduler.SchedulerBroker.send(SchedulerBroker.java:195) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.CompositeDestinationBroker.send(CompositeDestinationBroker.java:96) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.TransactionBroker.send(TransactionBroker.java:293) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:158) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.util.LoggingBrokerPlugin.send(LoggingBrokerPlugin.java:275) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.MutableBrokerFilter.send(MutableBrokerFilter.java:158) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.util.BrokerSupport.doResend(BrokerSupport.java:69) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.util.BrokerSupport.resendNoCopy(BrokerSupport.java:38) ~[activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.RegionBroker.sendToDeadLetterQueue(RegionBroker.java:799) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.BrokerFilter.sendToDeadLetterQueue(BrokerFilter.java:319) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.BrokerFilter.sendToDeadLetterQueue(BrokerFilter.java:319) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.BrokerFilter.sendToDeadLetterQueue(BrokerFilter.java:319) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.MutableBrokerFilter.sendToDeadLetterQueue(MutableBrokerFilter.java:331) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.util.LoggingBrokerPlugin.sendToDeadLetterQueue(LoggingBrokerPlugin.java:505) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.MutableBrokerFilter.sendToDeadLetterQueue(MutableBrokerFilter.java:331) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.PrefetchSubscription.sendToDLQ(PrefetchSubscription.java:535) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.PrefetchSubscription.acknowledge(PrefetchSubscription.java:394) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.AbstractRegion.acknowledge(AbstractRegion.java:526) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.region.RegionBroker.acknowledge(RegionBroker.java:484) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:88) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.BrokerFilter.acknowledge(BrokerFilter.java:88) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.TransactionBroker.acknowledge(TransactionBroker.java:276) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:98) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.util.LoggingBrokerPlugin.acknowledge(LoggingBrokerPlugin.java:162) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.MutableBrokerFilter.acknowledge(MutableBrokerFilter.java:98) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.TransportConnection.processMessageAck(TransportConnection.java:590) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.command.MessageAck.visit(MessageAck.java:245) [activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:336) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:200) [activemq-broker-5.14.3.jar:5.14.3]
              at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50) [activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:125) [activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:301) [activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83) [activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:233) [activemq-client-5.14.3.jar:5.14.3]
              at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215) [activemq-client-5.14.3.jar:5.14.3]
              at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
      Caused by: java.sql.SQLException: org.apache.commons.dbcp2.DelegatingPreparedStatement with address: "NULL" is closed.
              at org.apache.commons.dbcp2.DelegatingStatement.checkOpen(DelegatingStatement.java:122) ~[commons-dbcp2-2.1.1.jar:2.1.1]
              at org.apache.commons.dbcp2.DelegatingPreparedStatement.setString(DelegatingPreparedStatement.java:143) ~[commons-dbcp2-2.1.1.jar:2.1.1]
              at org.apache.activemq.store.jdbc.adapter.DefaultJDBCAdapter.doAddMessage(DefaultJDBCAdapter.java:233) ~[activemq-jdbc-store-5.14.3.jar:5.14.3]
              at org.apache.activemq.store.jdbc.JDBCMessageStore.addMessage(JDBCMessageStore.java:159) ~[activemq-jdbc-store-5.14.3.jar:5.14.3]
              ... 44 more
      
      Setup to reproduce:

      Configure jdbcPersistenceAdapter with a MySQL datasource and a individualDeadLetterStrategy:

      ...
      <policyEntry queue=">">
          <deadLetterStrategy>
              <individualDeadLetterStrategy queuePrefix="DLQ." useQueueForQueueMessages="true"/>
          </deadLetterStrategy>
      </policyEntry>
      ...
      <persistenceAdapter>
         <jdbcPersistenceAdapter dataDirectory="${activemq.data}" cleanupPeriod="0" dataSource="#mysql-ds"/>
      </persistenceAdapter>
      ...
      <bean id="mysql-ds" class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close">
        <property name="driverClassName" value="com.mysql.jdbc.Driver"/>
        <property name="url" value="jdbc:mysql://localhost/activemq?relaxAutoCommit=true"/>
        <property name="username" value="user"/>
        <property name="password" value="password"/>
        <property name="poolPreparedStatements" value="true"/>
      </bean>
      ...
      

      Setup two consumers and producers for two different queues, e.g test.queue.1 and test.queue.2

      The consumers should have no client-side redelivery and directly throw an exception when consuming the message to trigger the DLQ mechanism on the broker.

      Send around 1000 messages with each producer in a loop.

      After a few seconds you see the above mentioned SQLException in the ActiveMQ.log on level WARN.

      Attachments

        Activity

          People

            Unassigned Unassigned
            domaier Dominik Maier
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: