[AMQ-3240] load balance on a VirtualTopic queue broken where there are consumers subscribining to both the VT queue and VirtualTopic itself

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 5.4.2
• Fix Version/s: AGING_TO_DIE

Environment

Linux intel chip 64 bit
Java HotSpot 64-Bit Server

• Assignee: Unassigned
• Reporter: joe smith
• Created: 23/Mar/11 21:00
• Updated: 15/Aug/14 20:34
• Resolved: 15/Aug/14 20:34

Description

Thanks for any help. I apologize that I'm not familiar with the source code, but we plan to use this feature for production deployment.

We have a broker cluster of 4 brokers with minimum configuration. The problem seem to occur only in a clustered environment.

We have a virtual destination for both queue consumption as well as topic consumption. E.g: a producer sends to a virtual topic VirtualTopic.testtopic. One consumer group (3 consumers) subscribe to the VT as a queue: Consumer.group1.VirtualTopic.testtopic. Anther consumer group subscribes to the VirtualTopic directly: VirtualTopic.testtopic.

Where there is no topic consumer listening on VirtualTopic.testtopic, the messages are load-balanced across the queue consumers (listening on Consumer.group1.VirtualTopic.testtopic) which are connected to brokers in the cluster randomly. Working correctly. However, as soon as we connect one or more consumer subscribing to the virtual topic itself (VirtualTopic.testtopic), the number of messages received by queue consumer did not match what the producer sent. Sometimes the number fall short of what was sent. Most of the times the numbers received were 2 or 3 time more.

Also, sometimes when I shutdown and reconnect the queue consumers (when no more msgs were received), more messages were received again by the consumers on reconnect.

Checking the redelivered flag were 0. However, in checking the JMS Destination name for the queue consumers, it included both queue destination as well as topic destination. Not sure if that's by design. I assume the queue consumers should only receive messages addressed to the queue destination.

The topic consumers always got the exact number of messages sent by the producer.

Clients connect using failover protocol with list of broker host/port w/tcp transport. Not other parameters are used on the client provider uri.

Broker config is default out-of-the-box config, with the following changes:

Added:

<destinationInterceptors>
<virtualDestinationInterceptor>
<virtualDestinations>
<virtualTopic name="VirtualTopic.>" />
</virtualDestinations>
</virtualDestinationInterceptor>
</destinationInterceptors>

Added for clustering:

<networkConnectors>
<networkConnector uri="static:(tcp://host1:61616,tcp://host2:61616,tcp://host3:61616)"
conduitSubscriptions="false"
dynamicOnly="true"
networkTTL="4"
suppressDuplicateQueueSubscriptions="true"
>
<excludedDestinations>
<queue physicalName="Consumer.*.VirtualTopic.>"/>
</excludedDestinations>
</networkConnector>
</networkConnectors>

Modified for connection load-balancing:

<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616" updateClusterClients="true" rebalanceClusterClients="true" updateClusterClientsOnRemove="true"/>
</transportConnectors>

Everything else is default config. I also tried with an additional transport (port 62001) dedicated to the networkConnector. The same issue still occurred. Sometimes on client failover, the client were actually re-connected to the 62001 port. Not sure if this is also a problem.

Thanks you.