[ARTEMIS-2894] Message redistribution don't work anymore when paging start on cluster address

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.12.0, 2.13.0, 2.14.0, 2.15.0
• Fix Version/s: 2.16.0
• Assignee: Gary Tully
• Reporter: Gaetan Caumartin
• Created: 04/Sep/20 11:12
• Updated: 14/Jul/22 10:55
• Resolved: 25/Nov/20 16:22

Description

I was trying to reproduce the same network of broker architecture that I had with activemq brokers by using an artemis cluster.

Everything was ok in version 2.11.0, I was able to store millions of messages and to consume them later from another broker inside the same cluster.

Since the version 2.12.0 I encounter an issue. I'm still able to produce a huge quantity of messages thanks to the paging mechanism but when I try to consume them from another broker I'm not able to get all of them (most of them disappear).

 

I'm able to reproduce this issue like this on 2.12.0, 2.13.0, 2.14.0 and 2.15.0 versions:

• Start a cluster with 2 brokers (with ON_DEMAND, redistribution-delay=0 and with address full policy set with PAGE)
• Produce messages on a queue on the first broker until paging starts for this queue
  • ./artemis producer --destination AMQ.QUEUE --message-count 550000 --message-size 1000
  • from the logs: Starting paging on address 'AMQ.QUEUE'
    ; size is currently: 1,073,742,120 bytes; max-size-bytes: -1; global-size-bytes: 1,073,742,120 
• On the second broker, starts to consume messages on the same queue
  • ./artemis consumer --destination AMQ.QUEUE --message-count 550000
• The consumer receives a few messages and then it stops when the paging starts on the clustered address
  • Consumer still waiting for messages
  • from the logs: Starting paging on address '$.artemis.
    internal.sf.my-cluster.335036cc-edc0-11ea-bae1-005056a7672b'; size is currently: 2,259 bytes; max-size-bytes: -1; global-size-bytes: 1,073,744,379
  • messages move from AMQ.QUEUE to $.artemis.internal.sf.my-cluster.335036cc-edc0-11ea-bae1-005056a7672b and appeared as acknowledged on both queue of the first broker but doesn't appear on the second broker.
  • Finally the queues are empty and the consumer didn't receives the messages since the paging starts on the clustered address

I will continue to investigate this issue with more debug/trace.

 

Issue Links

• is related to: ARTEMIS-2768 Negative AddressSize in JMX