Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Cannot Reproduce
-
1.0.0
-
None
-
None
-
git tip of both proton and dispatch on debian testing
commit f7490003d3d88ee695cdbaaee887fb0c22a140a0 Author: Andrew Stitcher <astitcher@apache.org> Date: Fri May 19 09:54:00 2017 -0400 NO-JIRA: Ensure _GNU_SOURCE & _POSIX_C_SOURCE are not redefined
commit 8c9f4a581f7a62158d21bbe845edb3db60ae1d06 Author: Ganesh Murthy <gmurthy@redhat.com> Date: Tue May 16 11:25:39 2017 -0400 NO-JIRA - Added extra documentation for the logMessage field. Thank you Gordon Sim
git tip of both proton and dispatch on debian testing commit f7490003d3d88ee695cdbaaee887fb0c22a140a0 Author: Andrew Stitcher <astitcher@apache.org> Date: Fri May 19 09:54:00 2017 -0400 NO-JIRA: Ensure _GNU_SOURCE & _POSIX_C_SOURCE are not redefined commit 8c9f4a581f7a62158d21bbe845edb3db60ae1d06 Author: Ganesh Murthy <gmurthy@redhat.com> Date: Tue May 16 11:25:39 2017 -0400 NO-JIRA - Added extra documentation for the logMessage field. Thank you Gordon Sim
Description
Reproducible essentially every time with the following command (although it might take many iterations)
ctest -VV -R system_tests_two_routers --repeat-until-fail 1000
test 27 Start 27: system_tests_two_routers 27: Test command: /usr/bin/python "/main/qpid-dispatch/build/tests/run.py" "-m" "unittest" "-v" "system_tests_two_routers" 27: Test timeout computed to be: 1500 27: test_01_pre_settled (system_tests_two_routers.RouterTest) ... ok 27: test_02a_multicast_unsettled (system_tests_two_routers.RouterTest) ... ok 27: test_02c_sender_settles_first (system_tests_two_routers.RouterTest) ... ok 27: test_03_propagated_disposition (system_tests_two_routers.RouterTest) ... ok 27: test_04_unsettled_undeliverable (system_tests_two_routers.RouterTest) ... ok 27: test_05_three_ack (system_tests_two_routers.RouterTest) ... ok 27: test_08_message_annotations (system_tests_two_routers.RouterTest) ... ok 27: test_08a_strip_message_annotations_custom (system_tests_two_routers.RouterTest) ... ok 27: test_08a_test_strip_message_annotations_both_add_ingress_trace (system_tests_two_routers.RouterTest) ... ok 27: test_08a_test_strip_message_annotations_in (system_tests_two_routers.RouterTest) ... ok 27: test_08a_test_strip_message_annotations_no (system_tests_two_routers.RouterTest) ... ERROR 27: test_08a_test_strip_message_annotations_no_add_trace (system_tests_two_routers.RouterTest) ... ERROR 27: test_08a_test_strip_message_annotations_out (system_tests_two_routers.RouterTest) ... ERROR 27: test_08a_test_strip_message_annotations_out_custom (system_tests_two_routers.RouterTest) ... ERROR 27: test_09_management (system_tests_two_routers.RouterTest) ... recv: Connection refused 27: send: Broken pipe 27: FAIL 27: test_10_semantics_multicast (system_tests_two_routers.RouterTest) ... recv: Connection refused 27: send: Broken pipe 27: FAIL 27: test_11a_semantics_closest_is_local (system_tests_two_routers.RouterTest) ... recv: Connection refused 27: send: Broken pipe 27: FAIL 27: test_11b_semantics_closest_is_remote (system_tests_two_routers.RouterTest) ... recv: Connection refused 27: send: Broken pipe 27: FAIL 27: test_12_semantics_spread (system_tests_two_routers.RouterTest) ... recv: Connection refused 27: send: Broken pipe 27: FAIL 27: test_13_to_override (system_tests_two_routers.RouterTest) ... recv: Connection refused 27: send: Broken pipe 27: FAIL 27: test_14_excess_deliveries_released (system_tests_two_routers.RouterTest) ... ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: ERROR:root:proton:io: recv: Connection refused 27: FAIL 27: test_15_attach_on_inter_router (system_tests_two_routers.RouterTest) ... FAIL 27: ERROR 27: 27: ====================================================================== 27: ERROR: test_08a_test_strip_message_annotations_no (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 410, in test_08a_test_strip_message_annotations_no 27: self.routers[0].wait_address("message_annotations_strip_no/1", 0, 1) 27: File "/main/qpid-dispatch/tests/system_test.py", line 426, in wait_address 27: assert retry(check, **retry_kwargs) 27: File "/main/qpid-dispatch/tests/system_test.py", line 102, in retry 27: ret = function() 27: File "/main/qpid-dispatch/tests/system_test.py", line 421, in check 27: attribute_names=[u'name', u'subscriberCount', u'remoteCount']).get_entities() 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 221, in query 27: response = self.call(request) 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 162, in call 27: response = self.client.call(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 353, in call 27: self.sender.send(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 78, in send 27: self.connection.wait(lambda: _is_settled(delivery), msg="Sending on sender %s" % self.link.name, timeout=timeout) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 279, in wait 27: "Connection %s disconnected: %s" % (self.url, self.disconnected)) 27: ConnectionException: Connection amqp://0.0.0.0:27443/$management disconnected: Condition('proton:io', 'recv: Connection reset by peer (connection aborted) (connection aborted)') 27: 27: ====================================================================== 27: ERROR: test_08a_test_strip_message_annotations_no_add_trace (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 501, in test_08a_test_strip_message_annotations_no_add_trace 27: self.routers[0].wait_address("strip_message_annotations_no_add_trace/1", 0, 1) 27: File "/main/qpid-dispatch/tests/system_test.py", line 426, in wait_address 27: assert retry(check, **retry_kwargs) 27: File "/main/qpid-dispatch/tests/system_test.py", line 102, in retry 27: ret = function() 27: File "/main/qpid-dispatch/tests/system_test.py", line 421, in check 27: attribute_names=[u'name', u'subscriberCount', u'remoteCount']).get_entities() 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 221, in query 27: response = self.call(request) 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 162, in call 27: response = self.client.call(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 353, in call 27: self.sender.send(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 78, in send 27: self.connection.wait(lambda: _is_settled(delivery), msg="Sending on sender %s" % self.link.name, timeout=timeout) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 279, in wait 27: "Connection %s disconnected: %s" % (self.url, self.disconnected)) 27: ConnectionException: Connection amqp://0.0.0.0:27443/$management disconnected: Condition('proton:io', 'recv: Connection reset by peer (connection aborted) (connection aborted)') 27: 27: ====================================================================== 27: ERROR: test_08a_test_strip_message_annotations_out (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 598, in test_08a_test_strip_message_annotations_out 27: self.routers[0].wait_address("strip_message_annotations_out/1", 0, 1) 27: File "/main/qpid-dispatch/tests/system_test.py", line 426, in wait_address 27: assert retry(check, **retry_kwargs) 27: File "/main/qpid-dispatch/tests/system_test.py", line 102, in retry 27: ret = function() 27: File "/main/qpid-dispatch/tests/system_test.py", line 421, in check 27: attribute_names=[u'name', u'subscriberCount', u'remoteCount']).get_entities() 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 221, in query 27: response = self.call(request) 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 162, in call 27: response = self.client.call(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 353, in call 27: self.sender.send(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 78, in send 27: self.connection.wait(lambda: _is_settled(delivery), msg="Sending on sender %s" % self.link.name, timeout=timeout) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 279, in wait 27: "Connection %s disconnected: %s" % (self.url, self.disconnected)) 27: ConnectionException: Connection amqp://0.0.0.0:27443/$management disconnected: Condition('proton:io', 'recv: Connection reset by peer (connection aborted) (connection aborted)') 27: 27: ====================================================================== 27: ERROR: test_08a_test_strip_message_annotations_out_custom (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 637, in test_08a_test_strip_message_annotations_out_custom 27: self.routers[0].wait_address("strip_message_annotations_out/1", 0, 1) 27: File "/main/qpid-dispatch/tests/system_test.py", line 426, in wait_address 27: assert retry(check, **retry_kwargs) 27: File "/main/qpid-dispatch/tests/system_test.py", line 102, in retry 27: ret = function() 27: File "/main/qpid-dispatch/tests/system_test.py", line 421, in check 27: attribute_names=[u'name', u'subscriberCount', u'remoteCount']).get_entities() 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 221, in query 27: response = self.call(request) 27: File "/main/qpid-dispatch/python/qpid_dispatch/management/client.py", line 162, in call 27: response = self.client.call(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 353, in call 27: self.sender.send(request) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 78, in send 27: self.connection.wait(lambda: _is_settled(delivery), msg="Sending on sender %s" % self.link.name, timeout=timeout) 27: File "/usr/lib/python2.7/dist-packages/proton/utils.py", line 279, in wait 27: "Connection %s disconnected: %s" % (self.url, self.disconnected)) 27: ConnectionException: Connection amqp://0.0.0.0:27443/$management disconnected: Condition('proton:io', 'recv: Connection reset by peer (connection aborted) (connection aborted)') 27: 27: ====================================================================== 27: ERROR: tearDownClass (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_test.py", line 605, in tearDownClass 27: cls.tester.teardown() 27: File "/main/qpid-dispatch/tests/system_test.py", line 543, in teardown 27: raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) 27: RuntimeError: Errors during teardown: 27: 27: Process 621 error: exit code -6, expected -1 27: qdrouterd -c B.conf -I /main/qpid-dispatch/python 27: /main/qpid-dispatch/build/tests/system_test.dir/system_tests_two_routers/RouterTest/setUpClass/B-2.cmd 27: >>>> 27: qdrouterd: /main/qpid-dispatch/src/hash.c:94: qd_hash_internal_remove_item: Assertion `(bucket->items).size > 0' failed. 27: <<<< 27: 27: Process 616 error: exit code -11, expected -1 27: qdrouterd -c A.conf -I /main/qpid-dispatch/python 27: /main/qpid-dispatch/build/tests/system_test.dir/system_tests_two_routers/RouterTest/setUpClass/A-1.cmd 27: >>>> 27: <<<< 27: 27: ====================================================================== 27: FAIL: test_09_management (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 715, in test_09_management 27: sub = M.subscribe("amqp:/#") 27: File "/main/qpid-dispatch/tests/system_test.py", line 497, in subscribe 27: assert retry(lambda: subscription.address, **retry_args) # Wait for address 27: AssertionError 27: 27: ====================================================================== 27: FAIL: test_10_semantics_multicast (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 765, in test_10_semantics_multicast 27: M3.subscribe(addr) 27: File "/main/qpid-dispatch/tests/system_test.py", line 497, in subscribe 27: assert retry(lambda: subscription.address, **retry_args) # Wait for address 27: AssertionError 27: 27: ====================================================================== 27: FAIL: test_11a_semantics_closest_is_local (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 826, in test_11a_semantics_closest_is_local 27: M2.subscribe(addr) 27: File "/main/qpid-dispatch/tests/system_test.py", line 497, in subscribe 27: assert retry(lambda: subscription.address, **retry_args) # Wait for address 27: AssertionError 27: 27: ====================================================================== 27: FAIL: test_11b_semantics_closest_is_remote (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 891, in test_11b_semantics_closest_is_remote 27: M2.subscribe(addr) 27: File "/main/qpid-dispatch/tests/system_test.py", line 497, in subscribe 27: assert retry(lambda: subscription.address, **retry_args) # Wait for address 27: AssertionError 27: 27: ====================================================================== 27: FAIL: test_12_semantics_spread (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 945, in test_12_semantics_spread 27: M2.subscribe(addr) 27: File "/main/qpid-dispatch/tests/system_test.py", line 497, in subscribe 27: assert retry(lambda: subscription.address, **retry_args) # Wait for address 27: AssertionError 27: 27: ====================================================================== 27: FAIL: test_13_to_override (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 1013, in test_13_to_override 27: M2.subscribe(addr) 27: File "/main/qpid-dispatch/tests/system_test.py", line 497, in subscribe 27: assert retry(lambda: subscription.address, **retry_args) # Wait for address 27: AssertionError 27: 27: ====================================================================== 27: FAIL: test_14_excess_deliveries_released (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 1048, in test_14_excess_deliveries_released 27: self.assertEqual(None, test.error) 27: AssertionError: None != 'Timeout Expired' 27: 27: ====================================================================== 27: FAIL: test_15_attach_on_inter_router (system_tests_two_routers.RouterTest) 27: ---------------------------------------------------------------------- 27: Traceback (most recent call last): 27: File "/main/qpid-dispatch/tests/system_tests_two_routers.py", line 1054, in test_15_attach_on_inter_router 27: self.assertEqual(None, test.error) 27: AssertionError: None != 'Timeout Expired' 27: 27: ---------------------------------------------------------------------- 27: Ran 22 tests in 499.145s 27: 27: FAILED (failures=8, errors=5) 27/33 Test #27: system_tests_two_routers ..................***Failed 499.21 sec
gdb -c /main/qpid-dispatch/build/tests/system_test.dir/system_tests_two_routers/RouterTest/setUpClass/core `which qdrouterd` GNU gdb (Debian 7.12-6) 7.12.0.20161007-git Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/sbin/qdrouterd...(no debugging symbols found)...done. [New LWP 622] [New LWP 623] [New LWP 624] [New LWP 621] [New LWP 625] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `qdrouterd -c B.conf -I /main/qpid-dispatch/python'. Program terminated with signal SIGABRT, Aborted. #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. [Current thread is 1 (Thread 0x7f0682ea7700 (LWP 622))] (gdb) thread apply all bt Thread 5 (Thread 0x7f06816a4700 (LWP 625)): #0 0x00007f068570cf33 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84 #1 0x00007f0686583fcd in proactor_do_epoll (p=0x5606c21936c0, can_block=true) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1636 #2 0x00007f0686584185 in pn_proactor_wait (p=0x5606c21936c0) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1682 #3 0x00007f0686a25e7a in thread_run (arg=0x5606c21835e0) at /main/qpid-dispatch/src/server.c:817 #4 0x00007f0686367494 in start_thread (arg=0x7f06816a4700) at pthread_create.c:333 #5 0x00007f068570c93f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 Thread 4 (Thread 0x7f0686e57080 (LWP 621)): #0 0x00007f068570cf33 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84 #1 0x00007f0686583fcd in proactor_do_epoll (p=0x5606c21936c0, can_block=true) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1636 #2 0x00007f0686584185 in pn_proactor_wait (p=0x5606c21936c0) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1682 #3 0x00007f0686a25e7a in thread_run (arg=0x5606c21835e0) at /main/qpid-dispatch/src/server.c:817 #4 0x00007f0686a26987 in qd_server_run (qd=0x5606c1f06010) at /main/qpid-dispatch/src/server.c:1031 #5 0x00005606c16008d0 in ?? () #6 0x00005606c160125b in main () Thread 3 (Thread 0x7f0681ea5700 (LWP 624)): #0 0x00007f068570cf33 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84 #1 0x00007f0686583fcd in proactor_do_epoll (p=0x5606c21936c0, can_block=true) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1636 #2 0x00007f0686584185 in pn_proactor_wait (p=0x5606c21936c0) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1682 #3 0x00007f0686a25e7a in thread_run (arg=0x5606c21835e0) at /main/qpid-dispatch/src/server.c:817 #4 0x00007f0686367494 in start_thread (arg=0x7f0681ea5700) at pthread_create.c:333 #5 0x00007f068570c93f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 Thread 2 (Thread 0x7f06826a6700 (LWP 623)): #0 0x00007f068570cf33 in epoll_wait () at ../sysdeps/unix/syscall-template.S:84 #1 0x00007f0686583fcd in proactor_do_epoll (p=0x5606c21936c0, can_block=true) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1636 #2 0x00007f0686584185 in pn_proactor_wait (p=0x5606c21936c0) at /main/qpid-proton/proton-c/src/proactor/epoll.c:1682 ---Type <return> to continue, or q <return> to quit--- #3 0x00007f0686a25e7a in thread_run (arg=0x5606c21835e0) at /main/qpid-dispatch/src/server.c:817 #4 0x00007f0686367494 in start_thread (arg=0x7f06826a6700) at pthread_create.c:333 #5 0x00007f068570c93f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97 Thread 1 (Thread 0x7f0682ea7700 (LWP 622)): #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x00007f06856583fa in __GI_abort () at abort.c:89 #2 0x00007f068564fe37 in __assert_fail_base (fmt=<optimized out>, assertion=assertion@entry=0x7f0686a2bf1f "(bucket->items).size > 0", file=file@entry=0x7f0686a2bf00 "/main/qpid-dispatch/src/hash.c", line=line@entry=94, function=function@entry=0x7f0686a2bfe0 <__PRETTY_FUNCTION__.3317> "qd_hash_internal_remove_item") at assert.c:92 #3 0x00007f068564fee2 in __GI___assert_fail (assertion=0x7f0686a2bf1f "(bucket->items).size > 0", file=0x7f0686a2bf00 "/main/qpid-dispatch/src/hash.c", line=94, function=0x7f0686a2bfe0 <__PRETTY_FUNCTION__.3317> "qd_hash_internal_remove_item") at assert.c:101 #4 0x00007f06869f7e74 in qd_hash_internal_remove_item (h=0x7f067c000960, bucket=0x7f067c00c730, item=0x7f067c02c9e0, key=0x7f0682ea64a0) at /main/qpid-dispatch/src/hash.c:94 #5 0x00007f06869f8794 in qd_hash_remove_by_handle2 (h=0x7f067c000960, handle=0x7f067c030720, key=0x7f0682ea64a0) at /main/qpid-dispatch/src/hash.c:337 #6 0x00007f06869f8736 in qd_hash_remove_by_handle (h=0x7f067c000960, handle=0x7f067c030720) at /main/qpid-dispatch/src/hash.c:326 #7 0x00007f0686a180f9 in qdr_core_remove_address (core=0x5606c220a050, addr=0x7f067c028120) at /main/qpid-dispatch/src/router_core/router_core.c:310 #8 0x00007f0686a105e1 in qdr_check_addr_CT (core=0x5606c220a050, addr=0x7f067c028120, was_local=false) at /main/qpid-dispatch/src/router_core/connections.c:955 #9 0x00007f0686a12279 in qdr_link_inbound_detach_CT (core=0x5606c220a050, action=0x7f067400f9e0, discard=false) at /main/qpid-dispatch/src/router_core/connections.c:1617 #10 0x00007f0686a192e2 in router_core_thread (arg=0x5606c220a050) at /main/qpid-dispatch/src/router_core/router_core_thread.c:70 #11 0x00007f0686367494 in start_thread (arg=0x7f0682ea7700) at pthread_create.c:333 #12 0x00007f068570c93f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97