[PROTON-648] Memory leaks on aborted connections.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: proton-0.7, proton-0.8
• Fix Version/s: proton-0.8
• Component/s: proton-c
• Assignee: Rafael H. Schloming
• Reporter: Marcel Meulemans
• Created: 15/Aug/14 10:12
• Updated: 07/Jan/16 11:37
• Resolved: 29/Sep/14 17:35

Description

When a connection is aborted, open sessions can not be ended and attached links can not be not detached properly. Because of this there are some references to connection/sessions/links that are not released properly:

1. The session state does not release references to links in the local_handles/remote_handles hash map. This causes a leak because the session references links and the links reference the session but the engine does have a reference to any of these objects anymore.
2. The transport does not release references to sessions in the local_channels/remote_channels hash map. This is not really a leak because the references are released in pn_transport_free, but i.m.o. they should be release when the connection is unbound from the transport, i.e. pn_unbind_transport().

The leak can be observed by using the attached test program with ActiveMQ. When ActiveMQ is shutdown with CTRL-C it just kills the connection causing a transport error (connection aborted) on the client side. When this happens the program leaks. See below for the program/valgrind output of the program with and without the patch.

The patch can be found here:
https://github.com/marcelmeulemans/qpid-proton/commit/e5300da7be015ea874248b6593040d1bd50ac0e3

• The removed commented code in pn_session_finalize is the right idea in the wrong place; pn_session_finalize is never called because the references that this code releases still exist. The code has been moved to a new function pn_cleanup_session in the transport "class".
• The pn_cleanup_session call from pn_remove_session is needed in the case the session is freed (pn_session_free) manually from application code.
• The pn_cleanup_session call from pn_transport_unbind is needed in the case the session is automatically freed via the connection.
• The changes in pn_session_close solve the real leak describe in 1 above.

Valgrind output without patch:

==5607== Memcheck, a memory error detector
==5607== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==5607== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==5607== Command: ./leak.symbols
==5607== 
Connected to 127.0.0.1:5672
[0x6671b80]:  -> SASL
[0x6671b80]:  <- SASL
[0x6671b80]:0 <- @sasl-mechanisms(64) [sasl-server-mechanisms=@PN_SYMBOL[:ANONYMOUS, :PLAIN]]
[0x6671b80]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS, initial-response=b""]
[0x6671b80]:0 <- @sasl-outcome(68) [code=0]
[0x6671b80]:  <- AMQP
[0x6671b80]:  -> AMQP
[0x6671b80]:0 -> @open(16) [container-id="leak", hostname="leak"]
[0x6671b80]:0 -> @begin(17) [next-outgoing-id=0, incoming-window=2147483647, outgoing-window=0]
[0x6671b80]:0 -> @attach(18) [name="leak", handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=1, source=@source(40) [address="queue://leak", durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue://leak", durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
[0x6671b80]:0 -> @flow(19) [incoming-window=2147483647, next-outgoing-id=0, outgoing-window=0, handle=0, delivery-count=0, link-credit=8, drain=false]
[0x6671b80]:0 <- @open(16) [container-id="", hostname="", max-frame-size=1048576]
[0x6671b80]:0 <- @begin(17) [remote-channel=0, next-outgoing-id=1, incoming-window=0, outgoing-window=0, handle-max=1024]
[0x6671b80]:0 <- @attach(18) [name="leak", handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue://leak"], target=@target(41) [address="queue://leak"], incomplete-unsettled=false, initial-delivery-count=0]
[0x6671b80]:0 -> @close(24) [error=@error(29) [condition=:"amqp:connection:framing-error"]]
[0x6671b80]:ERROR amqp:connection:framing-error connection aborted
[0x6671b80]:ERROR[-2] connection aborted

[0x6671b80]:  <- EOS
[0x6671b80]:  -> EOS (-2) connection aborted
Closed 127.0.0.1:5672
close: Bad file descriptor
==5607== 
==5607== HEAP SUMMARY:
==5607==     in use at exit: 14,904 bytes in 306 blocks
==5607==   total heap usage: 508 allocs, 202 frees, 1,151,697 bytes allocated
==5607== 
==5607== 14,904 (256 direct, 14,648 indirect) bytes in 1 blocks are definitely lost in loss record 306 of 306
==5607==    at 0x4C2935B: malloc (vg_replace_malloc.c:270)
==5607==    by 0x4150AF: pn_new (object.c:41)
==5607==    by 0x404D23: pn_connection (engine.c:384)
==5607==    by 0x40383B: main (leak.c:22)
==5607== 
==5607== LEAK SUMMARY:
==5607==    definitely lost: 256 bytes in 1 blocks
==5607==    indirectly lost: 14,648 bytes in 305 blocks
==5607==      possibly lost: 0 bytes in 0 blocks
==5607==    still reachable: 0 bytes in 0 blocks
==5607==         suppressed: 0 bytes in 0 blocks
==5607== 
==5607== For counts of detected and suppressed errors, rerun with: -v
==5607== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2)

Valgrind output with patch:

==4163== Memcheck, a memory error detector
==4163== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==4163== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==4163== Command: ./leak.symbols
==4163== 
Connected to 127.0.0.1:5672
[0x6671b80]:  -> SASL
[0x6671b80]:0 -> @sasl-init(65) [mechanism=:ANONYMOUS, initial-response=b""]
[0x6671b80]:  <- SASL
[0x6671b80]:0 <- @sasl-mechanisms(64) [sasl-server-mechanisms=@PN_SYMBOL[:ANONYMOUS, :PLAIN]]
[0x6671b80]:0 <- @sasl-outcome(68) [code=0]
[0x6671b80]:  <- AMQP
[0x6671b80]:  -> AMQP
[0x6671b80]:0 -> @open(16) [container-id="leak", hostname="leak"]
[0x6671b80]:0 -> @begin(17) [next-outgoing-id=0, incoming-window=2147483647, outgoing-window=0]
[0x6671b80]:0 -> @attach(18) [name="leak", handle=0, role=true, snd-settle-mode=0, rcv-settle-mode=1, source=@source(40) [address="queue://leak", durable=0, timeout=0, dynamic=false], target=@target(41) [address="queue://leak", durable=0, timeout=0, dynamic=false], initial-delivery-count=0]
[0x6671b80]:0 -> @flow(19) [incoming-window=2147483647, next-outgoing-id=0, outgoing-window=0, handle=0, delivery-count=0, link-credit=8, drain=false]
[0x6671b80]:0 <- @open(16) [container-id="", hostname="", max-frame-size=1048576]
[0x6671b80]:0 <- @begin(17) [remote-channel=0, next-outgoing-id=1, incoming-window=0, outgoing-window=0, handle-max=1024]
[0x6671b80]:0 <- @attach(18) [name="leak", handle=0, role=false, snd-settle-mode=2, rcv-settle-mode=0, source=@source(40) [address="queue://leak"], target=@target(41) [address="queue://leak"], incomplete-unsettled=false, initial-delivery-count=0]
[0x6671b80]:0 -> @close(24) [error=@error(29) [condition=:"amqp:connection:framing-error"]]
[0x6671b80]:ERROR amqp:connection:framing-error connection aborted
[0x6671b80]:ERROR[-2] connection aborted

[0x6671b80]:  <- EOS
[0x6671b80]:  -> EOS (-2) connection aborted
Closed 127.0.0.1:5672
close: Bad file descriptor
==4163== 
==4163== HEAP SUMMARY:
==4163==     in use at exit: 0 bytes in 0 blocks
==4163==   total heap usage: 508 allocs, 508 frees, 1,151,697 bytes allocated
==4163== 
==4163== All heap blocks were freed -- no leaks are possible
==4163== 
==4163== For counts of detected and suppressed errors, rerun with: -v
==4163== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)

Issue Links

• is related to: PROTON-641 pn_connection_t leaked when links not closed