Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-3669

Pending producer with qMirror, messages are not spooled to disk

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.5.1
    • Fix Version/s: 5.7.0
    • Component/s: Broker, Message Store
    • Labels:
      None
    • Environment:

      local test on windows, happens also on linux remotely

      Description

      We fill a queue that is backed by a qmirror with data (persistent) that exceeds the configured memory limit.
      The producer (producerFlowControl="true" at queue and qmirror) will be throttled when the the limit is reached, no messages are spooled to disk.
      As an overall result, we are not able to write a lot of messages into amq.

      Configuration:

       
      	<destinationInterceptors>
      		<mirroredQueue copyMessage = "true" postfix=".qmirror" prefix=""/>
      	</destinationInterceptors>
      
      	<destinationPolicy>
      		<policyMap>
      		  <policyEntries>
      			<policyEntry topic=">" producerFlowControl="true" memoryLimit="2mb" />
      			<policyEntry queue="created.static.for.persistent" producerFlowControl="true" memoryLimit="1mb" />
      		  </policyEntries>
      		</policyMap>
      	</destinationPolicy> 
      
      	<destinations>
      		<queue physicalName="created.static.for.persistent" />
      	</destinations>
      
      	<persistenceAdapter>
      		<kahaDB directory="${activemq.base}/data/kahadb"/>
      	</persistenceAdapter>
      	
      	<systemUsage>
      		<systemUsage>
      			<memoryUsage><memoryUsage limit="10 mb" /></memoryUsage>
      			<storeUsage><storeUsage limit="100 mb"/></storeUsage>
      			<tempUsage><tempUsage limit="100 mb"/></tempUsage>
      		</systemUsage>
      	</systemUsage>
      	
      	<transportConnectors>
      		<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
      	</transportConnectors>
       

      Java test code:

       
          ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616");
          Connection connection = factory.createConnection("user", "pwd");
          connection.start();
          Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
          Destination destination = session.createQueue("created.static.for.persistent");
          MessageProducer producer = session.createProducer(destination);
          producer.setDeliveryMode(DeliveryMode.PERSISTENT);
          char[] m = new char[1024];
          Arrays.fill(m, 'x');
          // create some messages that have 1k each
          for (int i = 0; i < 12000; i++) {
            ActiveMQTextMessage message = new ActiveMQTextMessage();
            message.setText(new String(m));
            producer.send(message);
          }
          connection.stop();
          connection.close();
        }
      
      

      Expectation:

      Messages should be written to disk when the memory limit exceeds, all messages should be available within the queue/topic.

      Result:

      INFO | Usage Manager memory limit (2097152) reached for topic://created.static.for.persistent.qmirror. Producers will be throttled to the rate at which messages are removed from this destination to prevent flooding it. See http://activemq.apache.org/producer-flow-control.html for more info

      Store percent used : 10
      Memory percent used : 20
      Temp percent used : 0

      Interesting: The smaller flow control for the queue (1mb) does not seem to catch but the qmirror does (2mb).

        Attachments

        1. mirroredqueue.diffs
          8 kB
          Claudio Corsi

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                darius.schier Darius Schier
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: