Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.12
-
None
-
Linux, ARM platform. Limited hardware resources. Federation link over a cellular data connection.
Description
PROBLEM STATEMENT:
I have a durable queue push route configured to send messages to a central broker. The link between the source broker and the destination broker is a flaky cellular connection. When the link goes on and off it will very frequently cause the broker to seg fault and die.
STEPS TO REPRODUCE:
This is tricky to reproduce but it most often occurs when the physical data connection is just being brought up. i.e. the cell connection went down, qpid releases the connection....after a while the connection comes back up and qpid tries to reconnect possibly (not certain) when the connection has not been completely set up.
RESULTS:
The broker receives some sort of data frame before the connection is fully established and therefore throws and exception. During this process it appears there is another thread that accesses some pointer that is invalid and causes the program to die.
LOGS:
The log shows the following messages just before dieing:
Nov 14 15:43:35 mtcdp daemon.err qpidd[6790]: 2011-11-14 15:43:35
error Connection local:55764-remote:5672 closed by error: Connection
not yet open, invalid frame received.(501)
Nov 14 15:43:35 mtcdp daemon.err qpidd[6790]: 2011-11-14 15:43:35
error Connection local:55764-remote:5672 closed by error: Connection
not yet open, invalid frame received.(501)
Notice that the connection error message shows up twice, this seems suspicious to me. This occurs every time the broker dies.
I got a core dump from the seg fault (a couple of times) and this is the backtrace
Core was generated by `qpidd'.
Program terminated with signal 11, Segmentation fault.
#0 0x403fdc98 in qpid::SessionState::disableReceiverTracking() ()
from /usr/lib/libqpidcommon.so.2
(gdb) backtrace
#0 0x403fdc98 in qpid::SessionState::disableReceiverTracking() ()
from /usr/lib/libqpidcommon.so.2
#1 0x4010e8a8 in
qpid::broker::Bridge::create(qpid::broker::Connection&) () from
/usr/lib/libqpidbroker.so.2
#2 0x4017ed18 in qpid::broker::Link::ioThreadProcessing() () from
/usr/lib/libqpidbroker.so.2
#3 0x40180920 in ?? () from /usr/lib/libqpidbroker.so.2
Cannot access memory at address 0x2d74c0f8