Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
0.20
-
None
Description
Use case:
A C++ broker is providing a service to many customers. In some cases the broker is delivering high volumes of near real-time traffic to many clients. Occasionally a client stops accepting traffic and his delivery queue backs up. This puts the broker into a low memory situation and all users are affected.
Solution:
The organization that owns the broker could provide an "overflow backup service" that does several things:
1. Provides a local-to-the-broker, high-horsepower backup engine that sits waiting for consumers' queues to fill up.
2. When a consumer queue fills to some threshhold value then the backup engine inserts itself in the consumer's data path with a "rebind" command.
3. The consumer's traffic is now directed to a queue dedicated to the backup engine. The backup engine buffers the consumer data and plays it back into the original consumer queue as space becomes available.
This feature could be approximated with no support from inside the broker but it could not be done without dropping or duplicating messages during the rebinding phase.
With internal broker support the rebind operation could be cleanly executed and the producers and consumers would be unaware of any backup engine activity.
A detailed use case is available in attached SlowConsumerRebind_v0.3.pdf