[AMQ-3669] Pending producer with qMirror, messages are not spooled to disk

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.5.1
• Fix Version/s: 5.7.0
• Component/s: Broker, Message Store

Environment

local test on windows, happens also on linux remotely

• Assignee: Unassigned
• Reporter: Darius Schier
• Created: 20/Jan/12 09:37
• Updated: 02/Jun/17 09:35
• Resolved: 23/May/12 20:27

Description

We fill a queue that is backed by a qmirror with data (persistent) that exceeds the configured memory limit.
The producer (producerFlowControl="true" at queue and qmirror) will be throttled when the the limit is reached, no messages are spooled to disk.
As an overall result, we are not able to write a lot of messages into amq.

Configuration:

 
	<destinationInterceptors>
		<mirroredQueue copyMessage = "true" postfix=".qmirror" prefix=""/>
	</destinationInterceptors>

	<destinationPolicy>
		<policyMap>
		  <policyEntries>
			<policyEntry topic=">" producerFlowControl="true" memoryLimit="2mb" />
			<policyEntry queue="created.static.for.persistent" producerFlowControl="true" memoryLimit="1mb" />
		  </policyEntries>
		</policyMap>
	</destinationPolicy> 

	<destinations>
		<queue physicalName="created.static.for.persistent" />
	</destinations>

	<persistenceAdapter>
		<kahaDB directory="${activemq.base}/data/kahadb"/>
	</persistenceAdapter>
	
	<systemUsage>
		<systemUsage>
			<memoryUsage><memoryUsage limit="10 mb" /></memoryUsage>
			<storeUsage><storeUsage limit="100 mb"/></storeUsage>
			<tempUsage><tempUsage limit="100 mb"/></tempUsage>
		</systemUsage>
	</systemUsage>
	
	<transportConnectors>
		<transportConnector name="openwire" uri="tcp://0.0.0.0:61616"/>
	</transportConnectors>
 

Java test code:

 
    ActiveMQConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616");
    Connection connection = factory.createConnection("user", "pwd");
    connection.start();
    Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
    Destination destination = session.createQueue("created.static.for.persistent");
    MessageProducer producer = session.createProducer(destination);
    producer.setDeliveryMode(DeliveryMode.PERSISTENT);
    char[] m = new char[1024];
    Arrays.fill(m, 'x');
    // create some messages that have 1k each
    for (int i = 0; i < 12000; i++) {
      ActiveMQTextMessage message = new ActiveMQTextMessage();
      message.setText(new String(m));
      producer.send(message);
    }
    connection.stop();
    connection.close();
  }

Expectation:

Messages should be written to disk when the memory limit exceeds, all messages should be available within the queue/topic.

Result:

INFO | Usage Manager memory limit (2097152) reached for topic://created.static.for.persistent.qmirror. Producers will be throttled to the rate at which messages are removed from this destination to prevent flooding it. See http://activemq.apache.org/producer-flow-control.html for more info

Store percent used : 10
Memory percent used : 20
Temp percent used : 0

Interesting: The smaller flow control for the queue (1mb) does not seem to catch but the qmirror does (2mb).

Issue Links

• relates to: AMQ-6695 MirrorQueue - send a copy to the mirror topic instead of original message