[PROTON-2005] [proton-c] AMQP error if delivery is aborted with session_outgoing bytes > 0

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: proton-c-0.26.0, proton-c-0.27.0, proton-c-0.27.1, proton-c-0.28.0, proton-c-0.29.0, proton-c-0.30.0
• Fix Version/s: proton-c-0.31.0
• Component/s: proton-c
• Assignee: Clifford Jansen
• Reporter: Charles E. Rolke
• Created: 08/Feb/19 10:51
• Updated: 14/May/20 09:30
• Resolved: 22/Jan/20 11:33

Description

Running the test case in https://issues.apache.org/jira/browse/DISPATCH-1265

The self test is  a two-router network with a test sender sending to router A and a test receiver receiving from router B. The test sender is sending some number of bytes and then aborting the delivery. In the observations here router A is transmitting incorrect AMQP over the interrouter link to router B in response to the abort.

• A 400K byte message is being transmitted. 320K bytes have already gone over the wire
• The message is aborted. with  session_outgoing 84992
• Over the wire six transfers send the 84992 bytes with the same delivery-id and with abort=true

A dispatch log scraper trace of router A for the interrouter link to router B:

 

  delivery 21 is in progress
2019-02-08 05:07:41.140975 A0#1480 A [A0_4] -> [B0_1] B transfer [0,10] (21) more transfer_93 - 16346 bytes (pending)
2019-02-08 05:07:41.141066 A0#1486 A [A0_4] -> [B0_1] B transfer [0,10] (21) more transfer_99 - 16346 bytes (pending)
2019-02-08 05:07:41.141087 A0#1488 A [A0_4] -> [B0_1] B transfer [0,10] (21) more transfer_102 - 228 bytes (pending)
2019-02-08 05:07:41.141169 A0#1492 A [A0_4] <- [B0_1] B flow [0,10] (14,250)
  delivery 21 is aborted with session_outgoing 84992
  over the wire the session bytes are bled down with multiple transfers that have both 'more' and 'aborted' set true
2019-02-08 05:07:41.141514 A0#1513 A [A0_4] -> [B0_1] B transfer [0,10] (21) settled more aborted transfer_93 - 16342 bytes transfer presettled
2019-02-08 05:07:41.141538 A0#1514 A [A0_4] -> [B0_1] B transfer [0,10] (21) settled more aborted transfer_94 - 16342 bytes transfer presettled
2019-02-08 05:07:41.141564 A0#1517 A [A0_4] -> [B0_1] B transfer [0,10] (21) settled more aborted transfer_96 - 16342 bytes transfer presettled
2019-02-08 05:07:41.141603 A0#1521 A [A0_4] -> [B0_1] B transfer [0,10] (21) settled more aborted transfer_99 - 16342 bytes transfer presettled
2019-02-08 05:07:41.141669 A0#1527 A [A0_4] -> [B0_1] B transfer [0,10] (21) settled more aborted transfer_92 - 16342 bytes transfer presettled
2019-02-08 05:07:41.141722 A0#1531 A [A0_4] -> [B0_1] B transfer [0,10] (21) settled aborted transfer_93 - 13522 bytes transfer presettled
  delivery 22 is started
2019-02-08 05:07:41.142542 A0#1576 A [A0_4] -> [B0_1] B transfer [0,10] (22) more transfer_107 - 16346 bytes (pending)
2019-02-08 05:07:41.142578 A0#1579 A [A0_4] -> [B0_1] B transfer [0,10] (22) more transfer_108 - 16346 bytes (pending)
2019-02-08 05:07:41.142608 A0#1581 A [A0_4] -> [B0_1] B transfer [0,10] (22) more transfer_109 - 16346 bytes (pending) ...
  eventually peer router B closes the connection
2019-02-08 05:07:41.144746 A0#1730 A [A0_4] <- [B0_1] B close [0] error :"amqp:session:invalid-field" "sequencing error, expected delivery-id 22, got 21"
2019-02-08 05:07:41.144767 A0#1732 A [A0_4] <- [B0_1] B EOS

The transfer at 05:07:41.141514 settles the delivery and the next transfer is an AMQP protocol violation.

Issue Links

• causes: DISPATCH-1265 Delivery_abort test causes inter-router session error
• is related to: PROTON-2042 Occasional engine.c:691: pni_add_work: Assertion `!delivery->local.settled' failed
• is related to: PROTON-2128 Python client crash sending aborted messages