[PROTON-2101] local-idle-timeout is detected but on_transport_error callback is not called

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: proton-c-0.25.0, proton-c-0.26.0, proton-c-0.27.0, proton-c-0.27.1, proton-c-0.28.0, proton-c-0.29.0
• Component/s: cpp-binding, proton-c
• Environment: Windows, 64-bit, TLS, Visual Studio 17 C++
• Flags: Important
• Assignee: Unassigned
• Reporter: Bajan Fella
• Created: 17/Sep/19 13:51
• Updated: 02/Sep/21 09:56

Description

I am running 0.25 and I see an issue where the local-dle-timeout is being detected but the on_transport_error callback is not being called. As a result sometimes when my application experience temporary network issues it cannot recover. This is a release blocker for me. 

I have all my TRACE flags enabled and I am seeing

->[000001661BD6BDF0]:ERROR amqp:resource-limit-exceeded local-idle-timeout expired

 ->[000001661BD6BDF0]:0 -> @close(24) [error=@error(29) [condition=:"amqp:resource-limit-exceeded", description="local-idle-timeout expired"]]

When I caught this issue in the debugger I can see the PN_TRANSPORT_TAIL_CLOSED event but never receive the PN_TRANSPORT_CLOSED event.

To reproduce, I took the \qpid-proton-master\cpp\examples\simple_recv.cpp from the 0.29.0 repo and modified it so I can connect to my server which is setup for TLS.  Please see attached sample code.

Once I connect I then use Clumsy (https://jagt.github.io/clumsy/index.html) to slow down/throttle/drop some packets on the port. For example, if my server is listening on port pxxx at ipx.ipx.ipx.ipx. I use 

(((ip.DstAddr=ipx.ipx.ipx.ipx) and (tcp.DstPort=pxxx)) or ((ip.SrcAddr=ipx.ipx.ipx.ipx) and (tcp.SrcPort=pxxx))) as a filter in Clumsy After a minute our so I get the expected output in the trace and netstat -ano | finsstr /c:pxxx shows that the TCP connection is closed but I did not receive a transport_error or transport_close

[0000018B6B046210]:0 -> (EMPTY FRAME)
[0000018B6B046210]:RAW: "\x00\x00\x00\x08\x02\x00\x00\x00"
[0000018B6B046210]:0 <- (EMPTY FRAME)
[0000018B6B046210]:0 <- (EMPTY FRAME)
[0000018B6B046210]:0 -> (EMPTY FRAME)
[0000018B6B046210]:RAW: "\x00\x00\x00\x08\x02\x00\x00\x00"
[0000018B6B046210]:0 -> (EMPTY FRAME)
[0000018B6B046210]:RAW: "\x00\x00\x00\x08\x02\x00\x00\x00"
[0000018B6B046210]:ERROR amqp:resource-limit-exceeded local-idle-timeout expired
[0000018B6B046210]:0 -> @close(24) [error=@error(29) [condition=:"amqp:resource-limit-exceeded", description="local-idle-timeout expired"]]
[0000018B6B046210]:RAW: "\x00\x00\x00Z\x02\x00\x00\x00\x00S\x18\xd0\x00\x00\x00J\x00\x00\x00\x01\x00S\x1d\xd0\x00\x00\x00>\x00\x00\x00\x02\xa3\x1camqp:resource-limit-exceeded\xa1\x1alocal-idle-timeout expired"

 

Issue Links

• Testing discovered: PROTON-1890 [c++] implement idle_timeout and heartbeats