Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
1.9.0
-
None
-
None
Description
The router can grant extra credit during drain of a link-routed receiver, leading to the router having more outstanding credit to the peer than the receiver has actually granted.
The following shows a link-routed receiver draining credit from the link, after a single message having been received from the broker it was link-routed to. The router relays the flow from client to broker as expected, with "delivery-count=1, link-credit=999, drain=true", and relays the brokers response with "delivery-count=1000, link-credit=0, drain=true":
[0x27bb800]:1 <- @flow(19) [next-incoming-id=1, incoming-window=2047, next-outgoing-id=5, outgoing-window=2147483647, handle=2, delivery-count=1, link-credit=999, drain=true] [0x2763c90]:0 -> @flow(19) [next-incoming-id=2, incoming-window=2147483647, next-outgoing-id=4, outgoing-window=2147483647, handle=2, delivery-count=1, link-credit=999, drain=true] [0x2763c90]:0 <- @flow(19) [next-incoming-id=4, incoming-window=2147483647, next-outgoing-id=2, outgoing-window=2147483647, handle=2, delivery-count=1000, link-credit=0, drain=true] [0x27bb800]:1 -> @flow(19) [next-incoming-id=5, incoming-window=2147483647, next-outgoing-id=1, outgoing-window=2147483647, handle=2, delivery-count=1000, link-credit=0, drain=true]
The receiver then supplies more credit, which the router again relays as expected:
[0x27bb800]:1 <- @flow(19) [next-incoming-id=1, incoming-window=2047, next-outgoing-id=5, outgoing-window=2147483647, handle=2, delivery-count=1000, link-credit=1000] [0x2763c90]:0 -> @flow(19) [next-incoming-id=2, incoming-window=2147483647, next-outgoing-id=4, outgoing-window=2147483647, handle=2, delivery-count=1000, link-credit=1000, drain=false]
The receiver then drained this credit without having received any messages. The router relays the drain, but incorrectly grants additional credit to the broker at the same time, despite the receiver not having done so itself. The credit is drained, and the router updates the receiver to note it is also drained. Note however that the delivery counts have now diverged between client->router and router->broker, which seems unexpected for a link route.
[0x27bb800]:1 <- @flow(19) [next-incoming-id=1, incoming-window=2047, next-outgoing-id=7, outgoing-window=2147483647, handle=2, delivery-count=1000, link-credit=1000, drain=true] [0x2763c90]:0 -> @flow(19) [next-incoming-id=2, incoming-window=2147483647, next-outgoing-id=6, outgoing-window=2147483647, handle=2, delivery-count=1000, link-credit=2000, drain=true] [0x2763c90]:0 <- @flow(19) [next-incoming-id=6, incoming-window=2147483647, next-outgoing-id=2, outgoing-window=2147483647, handle=2, delivery-count=3000, link-credit=0, drain=true] [0x27bb800]:1 -> @flow(19) [next-incoming-id=7, incoming-window=2147483647, next-outgoing-id=1, outgoing-window=2147483647, handle=2, delivery-count=2000, link-credit=0, drain=true]
Had the broker been able to send messages using the mistakenly granted extra credit, the router would not be able to pass these all on to the client until it granted more credit. It isnt clear what would happen to them, but it seems like they could get stuck in the router until more credit is granted by the receiver, or the receiver detaches, etc. Again, unexpected for a link-route.
The system continued on in this case since since this receiver grants (and drains) new credit:
[0x27bb800]:1 <- @flow(19) [next-incoming-id=1, incoming-window=2047, next-outgoing-id=7, outgoing-window=2147483647, handle=2, delivery-count=2000, link-credit=1000] [0x2763c90]:0 -> @flow(19) [next-incoming-id=2, incoming-window=2147483647, next-outgoing-id=6, outgoing-window=2147483647, handle=2, delivery-count=3000, link-credit=1000, drain=false] etc etc
however the delivery-count divergence grows over time as the situation repeats.
Noticed while trying out changes for DISPATCH-1488.
Attachments
Attachments
Issue Links
- links to