[PROTON-2056] [proton-python] on_settled callback not called when disposition arrives in 2 frames

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: proton-c-0.28.0
• Fix Version/s: proton-c-0.29.0
• Component/s: proton-c, python-binding
• Assignee: Andrew Stitcher
• Reporter: Ganesh Murthy
• Created: 24/May/19 18:44
• Updated: 23/Apr/20 17:27
• Resolved: 19/Jul/19 19:34

Description

When very large anonymous messages are sent to the router and these messages have no receiver, they are immediately released. The router waits for the entire large message to arrive in the router before settling it. Due to this, in some cases, two disposition frames are sent for the same delivery, the first has state=released and the second has settled=true as seen below

 

0x56330c891430]:0 <- @disposition(21) [role=true, first=315, state=@released(38) []]
[0x56330c891430]:0 <- @disposition(21) [role=true, first=315, settled=true, state=@released(38) []]

 

When this case happens, the on_settled is not called for the python binding. The on_released is called. The on_settled must be called when a settlement arrives for every delivery. I observed this behavior in a python system test in Dispatch Router. The test called

test_51_anon_sender_mobile_address_large_msg_edge_to_edge_two_interior can be found in tests/system_tests_edge_router.py

The test does not fail all the time but when it does it is due to the on_settled not being called for deliveries that have this two part disposition.

 

I tried in vain to write a standalone python reproducer. I could not do it.

 

To run the specific system test run the following from the qpid-dispatch/build folder

 

 /usr/bin/python "/home/gmurthy/opensource/qpid-dispatch/build/tests/run.py" "-m" "unittest" "-v" "system_tests_edge_router.RouterTest.test_51_anon_sender_mobile_address_large_msg_edge_to_edge_two_interior"

 

The following is the test failure

test_51_anon_sender_mobile_address_large_msg_edge_to_edge_two_interior (system_tests_edge_router.RouterTest) ... FAIL

======================================================================
FAIL: test_51_anon_sender_mobile_address_large_msg_edge_to_edge_two_interior (system_tests_edge_router.RouterTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gmurthy/opensource/qpid-dispatch/tests/system_tests_edge_router.py", line 964, in test_51_anon_sender_mobile_address_large_msg_edge_to_edge_two_interior
    self.assertEqual(None, test.error)
AssertionError: None != u'Timeout Expired - n_sent=350 n_accepted=300 n_modified=0 n_released=48'

----------------------------------------------------------------------
Ran 1 test in 17.661s

FAILED (failures=1)

Issue Links

• is related to: PROTON-2186 On released callback invoked twice for same delivery tag
• is related to: PROTON-1914 [c] receiver cannot settle an incomplete incoming message
• is related to: PROTON-2061 [C++] Sender auto settlement should only happen after receiving a settlement from the receiver
• is related to: PROTON-2062 [ruby] Sender auto settlement should only happen after receiving a settlement from the receiver
• is related to: PROTON-2063 [go] Sender auto settlement should only happen after receiving a settlement from the receiver
• is related to: PROTON-2060 [docs] The documentation for the sender auto-settle option needs clarification