Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.35.0, 2.36.0
-
None
-
None
Description
Mirroring may not replicate messages when an operator uses the message move operation available from JMX or UI console.
Messages are not replicated after the process of moving messages has finished, even if the target queue of the move operation is replicated via mirroring.
The following issue has been seen on a configuration with two Artemis brokers with bi-directional (dual) mirroring enabled.
Setup:
- Artemis broker 1 "mirroring on" Artemis broker 2
- Artemis broker 2 "mirroring on" Artemis broker 1
Scenario:
If messages are injected on broker 1 "queue.A", messages are replicated on broker 2 "queue.A" via mirroring
- When the move operation to "queue.B" is performed on broker 1, messages are correctly replicated on broker 2 "queue.B" after the move
- When the move operation to "queue.B" is performed on broker 2, messages disappear from broker 1 "queue.A" (acks are received) but messages are not replicated on broker 1 "queue.B" after the move.
Test case:
An integration test case is provided in attachment. To run it, add test code into AMQPReplicaTest class under Artemis integration-tests package.
Possible cause:
The mirroring is adding "x-opt-amq-bkr-id" and "x-opt-amq-mr-id" properties/annotations on messages, both properties are kept by the message move operation.
I think that the message created by the move operation should be considered as a brand new message, removing these properties during the message move operation may solve the problem.
Logs:
The following log appears on broker 2 and could be linked to the problem
2024-07-08 12:02:06,069 TRACE [org.apache.activemq.artemis.protocol.amqp.connect.mirror.AMQPMirrorControllerSource] sendMessage::Message AMQPStandardMessage( [durable=true, messageID=95, address=testMessageMove.moved, size=303, scanningStatus=SCANNED, applicationProperties={i=7}, messageAnnotations={x-opt-jms-dest=0, x-opt-jms-msg-type=5, x-opt-ORIG-QUEUE=testMessageMove, x-opt-ORIG-ADDRESS=testMessageMove, x-opt-ORIG-MESSAGE-ID=58, x-opt-ORIG-ROUTING-TYPE=1}, properties=Properties{messageId=ID:f12bc95c-a09a-4c50-ad43-d55e592c7347:1:1:1-8, userId=null, to='testMessageMove.moved', subject='null', replyTo='null', correlationId=null, contentType=null, contentEncoding=null, absoluteExpiryTime=null, creationTime=Mon Jul 08 12:02:05 CEST 2024, groupId='null', groupSequence=null, replyToGroupId='null'}, extraProperties = TypedProperties[x-opt-amq-bkr-id=2018f725-3d11-11ef-88d6-00155d092ac8, _AMQ_ORIG_QUEUE=testMessageMove, _AMQ_AD=testMessageMove.moved, _AMQ_ORIG_ROUTING_TYPE=1, _AMQ_ORIG_ADDRESS=testMessageMove, x-opt-amq-mr-id=44, _AMQ_ORIG_MESSAGE_ID=58]] already belonged to the node, 2018f725-3d11-11ef-88d6-00155d092ac8, it won't circle send