Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
2.29.0
-
None
Description
Dual queue federation, where clusters federate in both direction can suffer from message flip flopping once the priority adjustment kicks in.
If there is a large backlog, the lower priority federation consumer is in play once all of the local consumer credit is exhausted and the backlog can drain to the other cluster.
If demand is low there, the process can repeat. limiting the rate of the federation consumer can help but it is not ideal b/c when there is no local demand, we want to have a high rate of migration.
A possible solution is to have the federation consumer manage its own credit and only flow messages when the local queue has capacity. Then flow a batch of messages, and await again that the local queue has capacity. In this way, there is no thundering herd effect, but there is also fast migration of messages once there is demand.
the consumerWindowSize=0 is already in play for consumer.receive calls and there is already a defaultConsumerWindowSize for an address. These can be combined to realise batchFederationOnCapacity semantics.
Attachments
Issue Links
- relates to
-
-
- Closed
-
- links to
