[AMQ-2214] Announcement of new temporary queue may lose race with message pointing to it in network of brokers.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 5.2.0
• Component/s: Broker
• Assignee: Unassigned
• Reporter: Joerg O. Henne
• Created: 21/Apr/09 10:42
• Updated: 21/Apr/09 13:26
• Resolved: 21/Apr/09 13:26

Description

Scenario:

• one well-known initiation queue is used by
• clients and servers to
• establish a private session using a pair of temporary queues:
  1. client creates temporary queue
  2. creates message with reply-to pointing to temporary queue
  3. sends message to initiation queue
  4. server receives initiation message
  5. creates temporary queue
  6. replys with handshake reply with reply-to pointing to second temporary queue
• clients repeatedly exercise this chat scenario
• communication is provided over a network of brokers which is established using auto discovery.

Problem:

When this scenario is executed over a network of brokers, sometimes the temporary queue has not yet been established thoughout the network of brokers when one of the partners already received a message with reply-to set to the temporary queue. This leads to exceptions like this:

javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:jh-xp-2-1250-1240304283337-2:0:378
	at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1597)
	at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:227)
	at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
	at Client.chat(Client.java:46)
	at Client.main(Client.java:78)

The problem seems to be heavily timing-dependant as it is very hard to reproduce with several brokers running on the same machine. However, it is more easily reproduced with brokers running in separate virtual machines (VM as in VMWare) or even distributed across real hardware.

Work-around

A work-around for this problem (and the proof that the temporary queue "springs into existence" shortly after the exception is thrown) is depicted in the following retry code:

int trys = 5;
while (true)
	try {
		MessageProducer cp = s.createProducer(rq);
		cp.send(s.createTextMessage(rr));
		break;
	} catch (InvalidDestinationException e) {
		if (--trys > 0) {
			logger.error("Got InvalidDestinationException - retrying " + trys, e);
			Thread.sleep(100);
			continue;
		} else
			throw e;
	}