Description
There are several memory leaks reported by ctest with valigrind, example below. The exact set of leaks is not always the same, so there is probably a race condition involved, but I get some leaks on pretty much every run.
Aside: we stared using the gcc/clang sanitizers with proton, they require special build flags but are way faster than valgrind's memcheck/helgrind with similar error-detection.
make && ctest [ 70%] Built target qpid-dispatch [ 78%] Built target unit_tests_size [ 85%] Built target qdrouterd [ 93%] Built target unit_tests [100%] Built target doc_gen Test project /home/aconway/dispatch/reldbg Start 1: unit_tests_size_10000 1/35 Test #1: unit_tests_size_10000 ..................... Passed 1.18 sec Start 2: unit_tests_size_512 2/35 Test #2: unit_tests_size_512 ....................... Passed 1.14 sec Start 3: unit_tests_size_10 3/35 Test #3: unit_tests_size_10 ........................ Passed 1.18 sec Start 4: unit_tests_size_7 4/35 Test #4: unit_tests_size_7 ......................... Passed 1.13 sec Start 5: unit_tests_size_5 5/35 Test #5: unit_tests_size_5 ......................... Passed 1.21 sec Start 6: unit_tests_size_3 6/35 Test #6: unit_tests_size_3 ......................... Passed 1.20 sec Start 7: unit_tests_size_2 7/35 Test #7: unit_tests_size_2 ......................... Passed 1.20 sec Start 8: unit_tests_size_1 8/35 Test #8: unit_tests_size_1 ......................... Passed 1.15 sec Start 9: unit_tests 9/35 Test #9: unit_tests ................................ Passed 4.47 sec Start 10: router_engine_test 10/35 Test #10: router_engine_test ........................ Passed 0.05 sec Start 11: management_test 11/35 Test #11: management_test ........................... Passed 0.04 sec Start 12: router_policy_test 12/35 Test #12: router_policy_test ........................ Passed 0.10 sec Start 13: system_tests_link_routes 13/35 Test #13: system_tests_link_routes ..................***Failed 24.66 sec test_aaa_qdmanage_query_link_route (system_tests_link_routes.LinkRouteTest) ... ok test_bbb_qdstat_link_routes_routerB (system_tests_link_routes.LinkRouteTest) ... FAIL test_ccc_qdstat_link_routes_routerC (system_tests_link_routes.LinkRouteTest) ... ok test_close_with_unsettled (system_tests_link_routes.LinkRouteTest) ... ok test_custom_annotations_match (system_tests_link_routes.LinkRouteTest) ... ok test_ddd_partial_link_route_match (system_tests_link_routes.LinkRouteTest) ... ok test_detach_mixed_close (system_tests_link_routes.LinkRouteTest) ... ok test_detach_without_close (system_tests_link_routes.LinkRouteTest) ... ok test_dynamic_source (system_tests_link_routes.LinkRouteTest) ... ok test_dynamic_target (system_tests_link_routes.LinkRouteTest) ... ok test_full_link_route_match (system_tests_link_routes.LinkRouteTest) ... ok test_full_link_route_match_1 (system_tests_link_routes.LinkRouteTest) ... ok test_link_route_pattern_match (system_tests_link_routes.LinkRouteTest) ... ok test_link_route_terminus_address (system_tests_link_routes.LinkRouteTest) ... ok test_partial_link_route_match_1 (system_tests_link_routes.LinkRouteTest) ... ok test_www_drain_support_all_messages (system_tests_link_routes.LinkRouteTest) ... ok test_www_drain_support_no_messages (system_tests_link_routes.LinkRouteTest) ... ok test_www_drain_support_no_more_messages (system_tests_link_routes.LinkRouteTest) ... ok test_www_drain_support_one_message (system_tests_link_routes.LinkRouteTest) ... ok test_yyy_delivery_tag (system_tests_link_routes.LinkRouteTest) ... ok test_zzz_qdmanage_delete_link_route (system_tests_link_routes.LinkRouteTest) ... FAIL ERROR ====================================================================== ERROR: tearDownClass (system_tests_link_routes.LinkRouteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30360 error: exit code 42, expected 0 qdrouterd -c D.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_link_routes/LinkRouteTest/setUpClass/D-4.cmd >>>> ==30360== Memcheck, a memory error detector ==30360== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30360== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30360== Command: qdrouterd -c D.conf -I /home/aconway/dispatch/python ==30360== ==30360== ==30360== HEAP SUMMARY: ==30360== in use at exit: 5,157,098 bytes in 29,291 blocks ==30360== total heap usage: 189,328 allocs, 160,037 frees, 27,698,972 bytes allocated ==30360== ==30360== 3,901 (2,368 direct, 1,533 indirect) bytes in 1 blocks are definitely lost in loss record 3,357 of 3,505 ==30360== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30360== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30360== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30360== by 0x4E83588: qd_server_connection (server.c:500) ==30360== by 0x4E84F4C: on_accept (server.c:531) ==30360== by 0x4E84F4C: handle_listener (server.c:698) ==30360== by 0x4E84F4C: handle (server.c:842) ==30360== by 0x4E85AAF: thread_run (server.c:919) ==30360== by 0x54EC36C: start_thread (pthread_create.c:456) ==30360== by 0x6064BBE: clone (clone.S:97) ==30360== ==30360== LEAK SUMMARY: ==30360== definitely lost: 2,368 bytes in 1 blocks ==30360== indirectly lost: 1,533 bytes in 21 blocks ==30360== possibly lost: 1,343,425 bytes in 5,298 blocks ==30360== still reachable: 3,809,772 bytes in 23,971 blocks ==30360== suppressed: 0 bytes in 0 blocks ==30360== Reachable blocks (those to which a pointer was found) are not shown. ==30360== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30360== ==30360== For counts of detected and suppressed errors, rerun with: -v ==30360== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) <<<< Process 30359 error: exit code 42, expected 0 qdrouterd -c C.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_link_routes/LinkRouteTest/setUpClass/C-3.cmd >>>> ==30359== Memcheck, a memory error detector ==30359== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30359== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30359== Command: qdrouterd -c C.conf -I /home/aconway/dispatch/python ==30359== ==30359== ==30359== HEAP SUMMARY: ==30359== in use at exit: 5,384,620 bytes in 30,627 blocks ==30359== total heap usage: 258,614 allocs, 227,987 frees, 35,695,476 bytes allocated ==30359== ==30359== 64 bytes in 1 blocks are definitely lost in loss record 1,051 of 3,557 ==30359== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30359== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30359== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30359== by 0x4E605A5: setup_incoming_link (container.c:134) ==30359== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30359== by 0x4E84DEC: handle (server.c:901) ==30359== by 0x4E85AAF: thread_run (server.c:919) ==30359== by 0x54EC36C: start_thread (pthread_create.c:456) ==30359== by 0x6064BBE: clone (clone.S:97) ==30359== ==30359== 64 bytes in 1 blocks are definitely lost in loss record 1,052 of 3,557 ==30359== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30359== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30359== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30359== by 0x4E605A5: setup_incoming_link (container.c:134) ==30359== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30359== by 0x4E84DEC: handle (server.c:901) ==30359== by 0x4E85AAF: thread_run (server.c:919) ==30359== by 0x4E85B5F: qd_server_run (server.c:1157) ==30359== by 0x4016AB: main_process (main.c:111) ==30359== by 0x40146E: main (main.c:318) ==30359== ==30359== 256 bytes in 4 blocks are definitely lost in loss record 2,355 of 3,557 ==30359== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30359== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30359== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30359== by 0x4E60099: setup_outgoing_link (container.c:99) ==30359== by 0x4E60099: qd_container_handle_event (container.c:447) ==30359== by 0x4E84DEC: handle (server.c:901) ==30359== by 0x4E85AAF: thread_run (server.c:919) ==30359== by 0x54EC36C: start_thread (pthread_create.c:456) ==30359== by 0x6064BBE: clone (clone.S:97) ==30359== ==30359== 57,283 (64 direct, 57,219 indirect) bytes in 1 blocks are definitely lost in loss record 3,548 of 3,557 ==30359== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30359== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30359== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30359== by 0x4E7AADA: qdr_add_link_ref (router_core.c:384) ==30359== by 0x4E74E76: qdr_link_enqueue_work_CT (connections.c:540) ==30359== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30359== by 0x54EC36C: start_thread (pthread_create.c:456) ==30359== by 0x6064BBE: clone (clone.S:97) ==30359== ==30359== LEAK SUMMARY: ==30359== definitely lost: 448 bytes in 7 blocks ==30359== indirectly lost: 57,219 bytes in 382 blocks ==30359== possibly lost: 1,433,710 bytes in 5,794 blocks ==30359== still reachable: 3,893,243 bytes in 24,444 blocks ==30359== suppressed: 0 bytes in 0 blocks ==30359== Reachable blocks (those to which a pointer was found) are not shown. ==30359== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30359== ==30359== For counts of detected and suppressed errors, rerun with: -v ==30359== ERROR SUMMARY: 4 errors from 4 contexts (suppressed: 0 from 0) <<<< Process 30358 error: exit code 42, expected 0 qdrouterd -c B.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_link_routes/LinkRouteTest/setUpClass/B-2.cmd >>>> ==30358== Memcheck, a memory error detector ==30358== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30358== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30358== Command: qdrouterd -c B.conf -I /home/aconway/dispatch/python ==30358== ==30358== ==30358== HEAP SUMMARY: ==30358== in use at exit: 5,398,863 bytes in 30,718 blocks ==30358== total heap usage: 285,383 allocs, 254,665 frees, 38,374,539 bytes allocated ==30358== ==30358== 64 bytes in 1 blocks are definitely lost in loss record 1,085 of 3,606 ==30358== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30358== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30358== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30358== by 0x4E605A5: setup_incoming_link (container.c:134) ==30358== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30358== by 0x4E84DEC: handle (server.c:901) ==30358== by 0x4E85AAF: thread_run (server.c:919) ==30358== by 0x4E85B5F: qd_server_run (server.c:1157) ==30358== by 0x4016AB: main_process (main.c:111) ==30358== by 0x40146E: main (main.c:318) ==30358== ==30358== 64 bytes in 1 blocks are definitely lost in loss record 1,086 of 3,606 ==30358== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30358== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30358== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30358== by 0x4E605A5: setup_incoming_link (container.c:134) ==30358== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30358== by 0x4E84DEC: handle (server.c:901) ==30358== by 0x4E85AAF: thread_run (server.c:919) ==30358== by 0x54EC36C: start_thread (pthread_create.c:456) ==30358== by 0x6064BBE: clone (clone.S:97) ==30358== ==30358== 128 bytes in 2 blocks are definitely lost in loss record 2,074 of 3,606 ==30358== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30358== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30358== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30358== by 0x4E624AE: qd_hash_internal_insert (hash.c:156) ==30358== by 0x4E624AE: qd_hash_insert (hash.c:168) ==30358== by 0x4E7A615: qdr_add_local_address_CT (router_core.c:309) ==30358== by 0x4E7C7F7: qdr_route_table_setup_CT (route_tables.c:229) ==30358== by 0x4E7AEBD: router_core_thread (router_core_thread.c:52) ==30358== by 0x54EC36C: start_thread (pthread_create.c:456) ==30358== by 0x6064BBE: clone (clone.S:97) ==30358== ==30358== 128 bytes in 2 blocks are definitely lost in loss record 2,075 of 3,606 ==30358== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30358== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30358== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30358== by 0x4E60099: setup_outgoing_link (container.c:99) ==30358== by 0x4E60099: qd_container_handle_event (container.c:447) ==30358== by 0x4E84DEC: handle (server.c:901) ==30358== by 0x4E85AAF: thread_run (server.c:919) ==30358== by 0x54EC36C: start_thread (pthread_create.c:456) ==30358== by 0x6064BBE: clone (clone.S:97) ==30358== ==30358== 320 (128 direct, 192 indirect) bytes in 1 blocks are definitely lost in loss record 2,688 of 3,606 ==30358== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30358== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30358== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30358== by 0x4E78C3C: qdr_route_declare_id_CT.isra.2 (route_control.c:65) ==30358== by 0x4E796C5: qdr_route_add_link_route_CT (route_control.c:334) ==30358== by 0x4E73212: qdra_config_link_route_create_CT (agent_config_link_route.c:449) ==30358== by 0x4E6F3E0: qdr_manage_create_CT (agent.c:375) ==30358== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30358== by 0x54EC36C: start_thread (pthread_create.c:456) ==30358== by 0x6064BBE: clone (clone.S:97) ==30358== ==30358== LEAK SUMMARY: ==30358== definitely lost: 512 bytes in 7 blocks ==30358== indirectly lost: 192 bytes in 2 blocks ==30358== possibly lost: 1,466,274 bytes in 6,008 blocks ==30358== still reachable: 3,931,885 bytes in 24,701 blocks ==30358== suppressed: 0 bytes in 0 blocks ==30358== Reachable blocks (those to which a pointer was found) are not shown. ==30358== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30358== ==30358== For counts of detected and suppressed errors, rerun with: -v ==30358== ERROR SUMMARY: 5 errors from 5 contexts (suppressed: 0 from 0) <<<< Process 30357 error: exit code 42, expected 0 qdrouterd -c A.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_link_routes/LinkRouteTest/setUpClass/A-1.cmd >>>> ==30357== Memcheck, a memory error detector ==30357== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30357== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30357== Command: qdrouterd -c A.conf -I /home/aconway/dispatch/python ==30357== ==30357== ==30357== HEAP SUMMARY: ==30357== in use at exit: 5,177,031 bytes in 29,420 blocks ==30357== total heap usage: 203,181 allocs, 173,761 frees, 29,730,755 bytes allocated ==30357== ==30357== 64 bytes in 1 blocks are definitely lost in loss record 1,079 of 3,544 ==30357== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30357== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30357== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30357== by 0x4E605A5: setup_incoming_link (container.c:134) ==30357== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30357== by 0x4E84DEC: handle (server.c:901) ==30357== by 0x4E85AAF: thread_run (server.c:919) ==30357== by 0x4E85B5F: qd_server_run (server.c:1157) ==30357== by 0x4016AB: main_process (main.c:111) ==30357== by 0x40146E: main (main.c:318) ==30357== ==30357== LEAK SUMMARY: ==30357== definitely lost: 64 bytes in 1 blocks ==30357== indirectly lost: 0 bytes in 0 blocks ==30357== possibly lost: 1,363,832 bytes in 5,423 blocks ==30357== still reachable: 3,813,135 bytes in 23,996 blocks ==30357== suppressed: 0 bytes in 0 blocks ==30357== Reachable blocks (those to which a pointer was found) are not shown. ==30357== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30357== ==30357== For counts of detected and suppressed errors, rerun with: -v ==30357== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) <<<< ====================================================================== FAIL: test_bbb_qdstat_link_routes_routerB (system_tests_link_routes.LinkRouteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_link_routes.py", line 258, in test_bbb_qdstat_link_routes_routerB self.assertEqual(len(parts), 12) AssertionError: 17 != 12 ====================================================================== FAIL: test_zzz_qdmanage_delete_link_route (system_tests_link_routes.LinkRouteTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_link_routes.py", line 592, in test_zzz_qdmanage_delete_link_route self.assertEquals(out.rstrip(), '[]') AssertionError: "==30563== Memcheck, a memory error detector\n==30563== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.\n==30563== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info\n==30563== Command: /home/aconway/dispatch/tools/qdmanage QUERY --type=linkRoute --bus amqp://0.0.0.0:29805 --indent=-1 --timeout 10.0\n==30563== \n[]" != '[]' ---------------------------------------------------------------------- Ran 21 tests in 24.614s FAILED (failures=2, errors=1) Start 14: system_tests_autolinks 14/35 Test #14: system_tests_autolinks ....................***Failed 8.57 sec test_01_autolink_attach (system_tests_autolinks.AutolinkTest) ... ok test_02_autolink_credit (system_tests_autolinks.AutolinkTest) ... ok test_03_autolink_sender (system_tests_autolinks.AutolinkTest) ... ok test_04_autolink_receiver (system_tests_autolinks.AutolinkTest) ... ok test_05_inter_container_transfer (system_tests_autolinks.AutolinkTest) ... ok test_06_manage_autolinks (system_tests_autolinks.AutolinkTest) ... ERROR:root:Connectivity to the peer container was lost ERROR:root:Connectivity to the peer container was lost ERROR:root:Connectivity to the peer container was lost ERROR:root:Connectivity to the peer container was lost ERROR:root:Connectivity to the peer container was lost ok test_07_autolink_attach_with_ext_addr (system_tests_autolinks.AutolinkTest) ... ok test_08_autolink_sender_with_ext_addr (system_tests_autolinks.AutolinkTest) ... ok test_09_autolink_receiver_with_ext_addr (system_tests_autolinks.AutolinkTest) ... ok ERROR ====================================================================== ERROR: tearDownClass (system_tests_autolinks.AutolinkTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30570 error: exit code 42, expected 0 qdrouterd -c test-router.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_autolinks/AutolinkTest/setUpClass/test-router-1.cmd >>>> ==30570== Memcheck, a memory error detector ==30570== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30570== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30570== Command: qdrouterd -c test-router.conf -I /home/aconway/dispatch/python ==30570== ==30570== ==30570== HEAP SUMMARY: ==30570== in use at exit: 5,116,375 bytes in 29,095 blocks ==30570== total heap usage: 347,179 allocs, 318,084 frees, 38,819,203 bytes allocated ==30570== ==30570== 64 bytes in 1 blocks are definitely lost in loss record 1,052 of 3,467 ==30570== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30570== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30570== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30570== by 0x4E624AE: qd_hash_internal_insert (hash.c:156) ==30570== by 0x4E624AE: qd_hash_insert (hash.c:168) ==30570== by 0x4E7BF72: qdr_subscribe_CT (route_tables.c:671) ==30570== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30570== by 0x54EC36C: start_thread (pthread_create.c:456) ==30570== by 0x6064BBE: clone (clone.S:97) ==30570== ==30570== 1,614 (128 direct, 1,486 indirect) bytes in 1 blocks are definitely lost in loss record 3,169 of 3,467 ==30570== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30570== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30570== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30570== by 0x4E78E86: qdr_route_add_auto_link_CT (route_control.c:397) ==30570== by 0x4E71E6D: qdra_config_auto_link_create_CT (agent_config_auto_link.c:423) ==30570== by 0x4E6F3C8: qdr_manage_create_CT (agent.c:376) ==30570== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30570== by 0x54EC36C: start_thread (pthread_create.c:456) ==30570== by 0x6064BBE: clone (clone.S:97) ==30570== ==30570== LEAK SUMMARY: ==30570== definitely lost: 192 bytes in 2 blocks ==30570== indirectly lost: 1,486 bytes in 21 blocks ==30570== possibly lost: 1,317,152 bytes in 5,195 blocks ==30570== still reachable: 3,797,545 bytes in 23,877 blocks ==30570== suppressed: 0 bytes in 0 blocks ==30570== Reachable blocks (those to which a pointer was found) are not shown. ==30570== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30570== ==30570== For counts of detected and suppressed errors, rerun with: -v ==30570== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) <<<< ---------------------------------------------------------------------- Ran 9 tests in 8.527s FAILED (errors=1) Start 15: system_tests_drain 15/35 Test #15: system_tests_drain ........................***Failed 5.93 sec test_drain_support_1_all_messages (system_tests_drain.DrainSupportTest) ... ok test_drain_support_2_one_message (system_tests_drain.DrainSupportTest) ... ok test_drain_support_3_no_messages (system_tests_drain.DrainSupportTest) ... ok test_drain_support_4_no_more_messages (system_tests_drain.DrainSupportTest) ... ok ERROR ====================================================================== ERROR: tearDownClass (system_tests_drain.DrainSupportTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30605 error: exit code 42, expected 0 qdrouterd -c test-router.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_drain/DrainSupportTest/setUpClass/test-router-1.cmd >>>> ==30605== Memcheck, a memory error detector ==30605== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30605== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30605== Command: qdrouterd -c test-router.conf -I /home/aconway/dispatch/python ==30605== ==30605== ==30605== HEAP SUMMARY: ==30605== in use at exit: 5,076,923 bytes in 28,829 blocks ==30605== total heap usage: 183,933 allocs, 155,104 frees, 26,697,410 bytes allocated ==30605== ==30605== 128 bytes in 2 blocks are definitely lost in loss record 2,005 of 3,444 ==30605== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30605== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30605== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30605== by 0x4E60099: setup_outgoing_link (container.c:99) ==30605== by 0x4E60099: qd_container_handle_event (container.c:447) ==30605== by 0x4E84DEC: handle (server.c:901) ==30605== by 0x4E85AAF: thread_run (server.c:919) ==30605== by 0x4E85B5F: qd_server_run (server.c:1157) ==30605== by 0x4016AB: main_process (main.c:111) ==30605== by 0x40146E: main (main.c:318) ==30605== ==30605== 128 bytes in 2 blocks are definitely lost in loss record 2,006 of 3,444 ==30605== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30605== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30605== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30605== by 0x4E60099: setup_outgoing_link (container.c:99) ==30605== by 0x4E60099: qd_container_handle_event (container.c:447) ==30605== by 0x4E84DEC: handle (server.c:901) ==30605== by 0x4E85AAF: thread_run (server.c:919) ==30605== by 0x54EC36C: start_thread (pthread_create.c:456) ==30605== by 0x6064BBE: clone (clone.S:97) ==30605== ==30605== LEAK SUMMARY: ==30605== definitely lost: 256 bytes in 4 blocks ==30605== indirectly lost: 0 bytes in 0 blocks ==30605== possibly lost: 1,307,835 bytes in 5,158 blocks ==30605== still reachable: 3,768,832 bytes in 23,667 blocks ==30605== suppressed: 0 bytes in 0 blocks ==30605== Reachable blocks (those to which a pointer was found) are not shown. ==30605== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30605== ==30605== For counts of detected and suppressed errors, rerun with: -v ==30605== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) <<<< ---------------------------------------------------------------------- Ran 4 tests in 5.885s FAILED (errors=1) Start 16: system_tests_management 16/35 Test #16: system_tests_management ...................***Failed 21.95 sec test_bad_query (system_tests_management.ManagementTest) Test that various badly formed queries get the proper response ... ok test_connection (system_tests_management.ManagementTest) Verify there is at least one connection ... ok test_create_config_address (system_tests_management.ManagementTest) ... ok test_create_config_address_pattern (system_tests_management.ManagementTest) ... ok test_create_listener (system_tests_management.ManagementTest) Create a new listener on a running router ... ok test_dummy (system_tests_management.ManagementTest) Test all operations on the dummy test entity ... ok test_entity_names (system_tests_management.ManagementTest) ... ok test_get_attributes (system_tests_management.ManagementTest) ... ok test_get_operations (system_tests_management.ManagementTest) ... ok test_get_schema (system_tests_management.ManagementTest) ... ok test_get_types (system_tests_management.ManagementTest) ... ok test_link (system_tests_management.ManagementTest) Verify we can find our own reply-to address in links ... ok test_log (system_tests_management.ManagementTest) Create, update and query log entities ... ok test_query_attributes (system_tests_management.ManagementTest) Query with attributes only ... ok test_query_type (system_tests_management.ManagementTest) Query with type only ... ok test_query_type_attributes (system_tests_management.ManagementTest) Query with type and attribute names ... ok test_remote_node (system_tests_management.ManagementTest) Test that we can access management info of remote nodes using get_mgmt_nodes addresses ... ok test_router (system_tests_management.ManagementTest) Verify router counts match entity counts ... ok test_router_node (system_tests_management.ManagementTest) Test node entity in a trio of linked routers ... ok test_standalone_no_inter_router (system_tests_management.ManagementTest) Verify that we do not allow inter-router connectors or listeners in standalone mode ... ok ERROR ====================================================================== ERROR: tearDownClass (system_tests_management.ManagementTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30631 error: exit code 42, expected 0 qdrouterd -c router0.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_management/ManagementTest/setUpClass/router0-2.cmd >>>> ==30631== Memcheck, a memory error detector ==30631== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30631== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30631== Command: qdrouterd -c router0.conf -I /home/aconway/dispatch/python ==30631== ==30631== ==30631== HEAP SUMMARY: ==30631== in use at exit: 5,309,226 bytes in 30,168 blocks ==30631== total heap usage: 228,468 allocs, 198,300 frees, 32,594,244 bytes allocated ==30631== ==30631== 13,908 (64 direct, 13,844 indirect) bytes in 1 blocks are definitely lost in loss record 3,493 of 3,544 ==30631== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30631== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30631== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30631== by 0x4E7AADA: qdr_add_link_ref (router_core.c:384) ==30631== by 0x4E758CF: qdr_link_inbound_first_attach_CT (connections.c:1336) ==30631== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30631== by 0x54EC36C: start_thread (pthread_create.c:456) ==30631== by 0x6064BBE: clone (clone.S:97) ==30631== ==30631== LEAK SUMMARY: ==30631== definitely lost: 64 bytes in 1 blocks ==30631== indirectly lost: 13,844 bytes in 92 blocks ==30631== possibly lost: 1,421,695 bytes in 5,755 blocks ==30631== still reachable: 3,873,623 bytes in 24,320 blocks ==30631== suppressed: 0 bytes in 0 blocks ==30631== Reachable blocks (those to which a pointer was found) are not shown. ==30631== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30631== ==30631== For counts of detected and suppressed errors, rerun with: -v ==30631== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) <<<< Process 30630 error: exit code 42, expected 0 qdrouterd -c solo.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_management/ManagementTest/setUpClass/solo-1.cmd >>>> ==30630== Memcheck, a memory error detector ==30630== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30630== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30630== Command: qdrouterd -c solo.conf -I /home/aconway/dispatch/python ==30630== ==30630== ==30630== HEAP SUMMARY: ==30630== in use at exit: 5,279,181 bytes in 30,366 blocks ==30630== total heap usage: 330,323 allocs, 299,957 frees, 46,388,082 bytes allocated ==30630== ==30630== 64 bytes in 1 blocks are definitely lost in loss record 1,035 of 3,495 ==30630== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30630== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30630== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30630== by 0x4E605A5: setup_incoming_link (container.c:134) ==30630== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30630== by 0x4E84DEC: handle (server.c:901) ==30630== by 0x4E85AAF: thread_run (server.c:919) ==30630== by 0x54EC36C: start_thread (pthread_create.c:456) ==30630== by 0x6064BBE: clone (clone.S:97) ==30630== ==30630== 192 bytes in 3 blocks are definitely lost in loss record 2,187 of 3,495 ==30630== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30630== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30630== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30630== by 0x4E605A5: setup_incoming_link (container.c:134) ==30630== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30630== by 0x4E84DEC: handle (server.c:901) ==30630== by 0x4E85AAF: thread_run (server.c:919) ==30630== by 0x4E85B5F: qd_server_run (server.c:1157) ==30630== by 0x4016AB: main_process (main.c:111) ==30630== by 0x40146E: main (main.c:318) ==30630== ==30630== LEAK SUMMARY: ==30630== definitely lost: 256 bytes in 4 blocks ==30630== indirectly lost: 0 bytes in 0 blocks ==30630== possibly lost: 1,394,321 bytes in 5,693 blocks ==30630== still reachable: 3,884,604 bytes in 24,669 blocks ==30630== suppressed: 0 bytes in 0 blocks ==30630== Reachable blocks (those to which a pointer was found) are not shown. ==30630== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30630== ==30630== For counts of detected and suppressed errors, rerun with: -v ==30630== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) <<<< ---------------------------------------------------------------------- Ran 20 tests in 21.910s FAILED (errors=1) Start 17: system_tests_one_router 17/35 Test #17: system_tests_one_router ...................***Failed 37.61 sec test_01_pre_settled (system_tests_one_router.OneRouterTest) ... ok test_02a_multicast_unsettled (system_tests_one_router.OneRouterTest) ... ok test_02b_disp_to_closed_connection (system_tests_one_router.OneRouterTest) ... ok test_02c_sender_settles_first (system_tests_one_router.OneRouterTest) ... ok test_03_propagated_disposition (system_tests_one_router.OneRouterTest) ... ok test_04_unsettled_undeliverable (system_tests_one_router.OneRouterTest) ... ok test_05_three_ack (system_tests_one_router.OneRouterTest) ... ok test_08_message_annotations (system_tests_one_router.OneRouterTest) ... ok test_08a_strip_message_annotations_custom (system_tests_one_router.OneRouterTest) ... ok test_08a_test_strip_message_annotations_both (system_tests_one_router.OneRouterTest) ... ok test_08a_test_strip_message_annotations_both_custom (system_tests_one_router.OneRouterTest) ... ok test_08a_test_strip_message_annotations_in (system_tests_one_router.OneRouterTest) ... ok test_08a_test_strip_message_annotations_no (system_tests_one_router.OneRouterTest) ... ok test_08a_test_strip_message_annotations_no_add_trace (system_tests_one_router.OneRouterTest) ... ok test_08a_test_strip_message_annotations_out (system_tests_one_router.OneRouterTest) ... ok test_09_management (system_tests_one_router.OneRouterTest) ... ok test_09a_management_no_reply (system_tests_one_router.OneRouterTest) ... ok test_09c_management_get_operations (system_tests_one_router.OneRouterTest) ... ok test_09d_management_not_implemented (system_tests_one_router.OneRouterTest) ... ok test_10_semantics_multicast (system_tests_one_router.OneRouterTest) ... ok test_11_semantics_closest (system_tests_one_router.OneRouterTest) ... ok test_12_semantics_spread (system_tests_one_router.OneRouterTest) ... ok test_13_to_override (system_tests_one_router.OneRouterTest) ... ok test_14_send_settle_mode_settled (system_tests_one_router.OneRouterTest) ... ok test_15_excess_deliveries_released (system_tests_one_router.OneRouterTest) ... ok test_16_multicast_unsettled (system_tests_one_router.OneRouterTest) ... ok test_16a_multicast_no_receivcer (system_tests_one_router.OneRouterTest) ... ok test_18_released_vs_modified (system_tests_one_router.OneRouterTest) ... ok test_19_appearance_of_balance (system_tests_one_router.OneRouterTest) ... ok test_20_batched_settlement (system_tests_one_router.OneRouterTest) ... ok test_21_presettled_overflow (system_tests_one_router.OneRouterTest) ... ok test_22_large_streaming_test (system_tests_one_router.OneRouterTest) ... ok test_25_reject_coordinator (system_tests_one_router.OneRouterTest) ... FAIL test_27_create_unavailable_sender (system_tests_one_router.OneRouterTest) ... ok test_28_create_unavailable_receiver (system_tests_one_router.OneRouterTest) ... ok test_connection_properties (system_tests_one_router.OneRouterTest) ... ok test_listen_error (system_tests_one_router.OneRouterTest) Make sure a router exits if a initial listener fails, doesn't hang ... FAIL test_reject_disposition (system_tests_one_router.OneRouterTest) ... ok ERROR ====================================================================== ERROR: tearDownClass (system_tests_one_router.OneRouterTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30723 error: exit code 42, expected 0 qdrouterd -c test-router.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_one_router/OneRouterTest/setUpClass/test-router-1.cmd >>>> ==30723== Memcheck, a memory error detector ==30723== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30723== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30723== Command: qdrouterd -c test-router.conf -I /home/aconway/dispatch/python ==30723== ==30723== ==30723== HEAP SUMMARY: ==30723== in use at exit: 6,243,077 bytes in 32,008 blocks ==30723== total heap usage: 451,927 allocs, 419,919 frees, 177,638,759 bytes allocated ==30723== ==30723== 768 bytes in 12 blocks are definitely lost in loss record 3,006 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E605A5: setup_incoming_link (container.c:134) ==30723== by 0x4E605A5: qd_container_handle_event (container.c:455) ==30723== by 0x4E84DEC: handle (server.c:901) ==30723== by 0x4E85AAF: thread_run (server.c:919) ==30723== by 0x4E85B5F: qd_server_run (server.c:1157) ==30723== by 0x4016AB: main_process (main.c:111) ==30723== by 0x40146E: main (main.c:318) ==30723== ==30723== 2,560 (128 direct, 2,432 indirect) bytes in 1 blocks are definitely lost in loss record 3,276 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E7D56B: qdr_terminus (terminus.c:44) ==30723== by 0x4E80904: AMQP_outgoing_link_handler (router_node.c:602) ==30723== by 0x4E84DEC: handle (server.c:901) ==30723== by 0x4E85AAF: thread_run (server.c:919) ==30723== by 0x54EC36C: start_thread (pthread_create.c:456) ==30723== by 0x6064BBE: clone (clone.S:97) ==30723== ==30723== 2,624 bytes in 41 blocks are definitely lost in loss record 3,278 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E60099: setup_outgoing_link (container.c:99) ==30723== by 0x4E60099: qd_container_handle_event (container.c:447) ==30723== by 0x4E84DEC: handle (server.c:901) ==30723== by 0x4E85AAF: thread_run (server.c:919) ==30723== by 0x54EC36C: start_thread (pthread_create.c:456) ==30723== by 0x6064BBE: clone (clone.S:97) ==30723== ==30723== 4,604 (2,368 direct, 2,236 indirect) bytes in 1 blocks are definitely lost in loss record 3,393 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E83588: qd_server_connection (server.c:500) ==30723== by 0x4E84F4C: on_accept (server.c:531) ==30723== by 0x4E84F4C: handle_listener (server.c:698) ==30723== by 0x4E84F4C: handle (server.c:842) ==30723== by 0x4E85AAF: thread_run (server.c:919) ==30723== by 0x54EC36C: start_thread (pthread_create.c:456) ==30723== by 0x6064BBE: clone (clone.S:97) ==30723== ==30723== 170,624 (13,760 direct, 156,864 indirect) bytes in 43 blocks are definitely lost in loss record 3,538 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E7DE29: qdr_link_deliver (transfer.c:47) ==30723== by 0x4E818A6: AMQP_rx_handler (router_node.c:453) ==30723== by 0x4E6043E: do_receive (container.c:175) ==30723== by 0x4E6043E: qd_container_handle_event (container.c:523) ==30723== by 0x4E84DEC: handle (server.c:901) ==30723== by 0x4E85AAF: thread_run (server.c:919) ==30723== by 0x4E85B5F: qd_server_run (server.c:1157) ==30723== by 0x4016AB: main_process (main.c:111) ==30723== by 0x40146E: main (main.c:318) ==30723== ==30723== 174,592 (14,080 direct, 160,512 indirect) bytes in 44 blocks are definitely lost in loss record 3,539 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E77098: qdr_forward_new_delivery_CT (forwarder.c:104) ==30723== by 0x4E77ADA: qdr_forward_multicast_CT (forwarder.c:275) ==30723== by 0x4E7EDC0: qdr_link_forward_CT (transfer.c:758) ==30723== by 0x4E7F493: qdr_link_deliver_CT (transfer.c:912) ==30723== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30723== by 0x54EC36C: start_thread (pthread_create.c:456) ==30723== by 0x6064BBE: clone (clone.S:97) ==30723== ==30723== 178,560 (14,400 direct, 164,160 indirect) bytes in 45 blocks are definitely lost in loss record 3,540 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E77098: qdr_forward_new_delivery_CT (forwarder.c:104) ==30723== by 0x4E774EB: qdr_forward_balanced_CT (forwarder.c:657) ==30723== by 0x4E7EDC0: qdr_link_forward_CT (transfer.c:758) ==30723== by 0x4E7F493: qdr_link_deliver_CT (transfer.c:912) ==30723== by 0x4E7B0C3: router_core_thread (router_core_thread.c:83) ==30723== by 0x54EC36C: start_thread (pthread_create.c:456) ==30723== by 0x6064BBE: clone (clone.S:97) ==30723== ==30723== 468,224 (37,760 direct, 430,464 indirect) bytes in 118 blocks are definitely lost in loss record 3,543 of 3,545 ==30723== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30723== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30723== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30723== by 0x4E7DE29: qdr_link_deliver (transfer.c:47) ==30723== by 0x4E818A6: AMQP_rx_handler (router_node.c:453) ==30723== by 0x4E6043E: do_receive (container.c:175) ==30723== by 0x4E6043E: qd_container_handle_event (container.c:523) ==30723== by 0x4E84DEC: handle (server.c:901) ==30723== by 0x4E85AAF: thread_run (server.c:919) ==30723== by 0x54EC36C: start_thread (pthread_create.c:456) ==30723== by 0x6064BBE: clone (clone.S:97) ==30723== ==30723== LEAK SUMMARY: ==30723== definitely lost: 85,888 bytes in 305 blocks ==30723== indirectly lost: 916,668 bytes in 1,823 blocks ==30723== possibly lost: 1,379,319 bytes in 5,663 blocks ==30723== still reachable: 3,861,202 bytes in 24,217 blocks ==30723== suppressed: 0 bytes in 0 blocks ==30723== Reachable blocks (those to which a pointer was found) are not shown. ==30723== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30723== ==30723== For counts of detected and suppressed errors, rerun with: -v ==30723== ERROR SUMMARY: 8 errors from 8 contexts (suppressed: 0 from 0) <<<< ====================================================================== FAIL: test_25_reject_coordinator (system_tests_one_router.OneRouterTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_one_router.py", line 1152, in test_25_reject_coordinator self.assertTrue(test.passed) AssertionError: False is not true ====================================================================== FAIL: test_listen_error (system_tests_one_router.OneRouterTest) Make sure a router exits if a initial listener fails, doesn't hang ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_one_router.py", line 68, in test_listen_error self.assertEqual(1, r.wait()) AssertionError: 1 != 42 ---------------------------------------------------------------------- Ran 38 tests in 37.567s FAILED (failures=2, errors=1) Start 18: system_tests_handle_failover 18/35 Test #18: system_tests_handle_failover ..............***Failed 22.32 sec test_connector_has_failover_list (system_tests_handle_failover.FailoverTest) ... ERROR test_remove_router_B (system_tests_handle_failover.FailoverTest) ... ERROR ERROR ====================================================================== ERROR: test_connector_has_failover_list (system_tests_handle_failover.FailoverTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_handle_failover.py", line 115, in test_connector_has_failover_list output = json.loads(self.run_qdmanage(query_command)) File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded ====================================================================== ERROR: test_remove_router_B (system_tests_handle_failover.FailoverTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_handle_failover.py", line 133, in test_remove_router_B output = json.loads(self.run_qdmanage(query_command, address=self.routers[1].addresses[0])) File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded ====================================================================== ERROR: tearDownClass (system_tests_handle_failover.FailoverTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30914 error: exit code 42, expected 0 qdrouterd -c QDR.C.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_handle_failover/FailoverTest/setUpClass/QDR.C-3.cmd >>>> ==30914== Memcheck, a memory error detector ==30914== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30914== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30914== Command: qdrouterd -c QDR.C.conf -I /home/aconway/dispatch/python ==30914== ==30914== ==30914== HEAP SUMMARY: ==30914== in use at exit: 5,165,313 bytes in 29,347 blocks ==30914== total heap usage: 184,543 allocs, 155,196 frees, 26,403,128 bytes allocated ==30914== ==30914== 6,111 (2,368 direct, 3,743 indirect) bytes in 1 blocks are definitely lost in loss record 3,409 of 3,512 ==30914== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30914== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30914== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30914== by 0x4E83588: qd_server_connection (server.c:500) ==30914== by 0x4E84F4C: on_accept (server.c:531) ==30914== by 0x4E84F4C: handle_listener (server.c:698) ==30914== by 0x4E84F4C: handle (server.c:842) ==30914== by 0x4E85AAF: thread_run (server.c:919) ==30914== by 0x54EC36C: start_thread (pthread_create.c:456) ==30914== by 0x6064BBE: clone (clone.S:97) ==30914== ==30914== LEAK SUMMARY: ==30914== definitely lost: 2,368 bytes in 1 blocks ==30914== indirectly lost: 3,743 bytes in 38 blocks ==30914== possibly lost: 1,343,145 bytes in 5,305 blocks ==30914== still reachable: 3,816,057 bytes in 24,003 blocks ==30914== suppressed: 0 bytes in 0 blocks ==30914== Reachable blocks (those to which a pointer was found) are not shown. ==30914== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30914== ==30914== For counts of detected and suppressed errors, rerun with: -v ==30914== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) <<<< Process 30894 error: exit code 42, expected 0 qdrouterd -c QDR.A.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_handle_failover/FailoverTest/setUpClass/QDR.A-2.cmd >>>> ==30894== Memcheck, a memory error detector ==30894== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30894== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30894== Command: qdrouterd -c QDR.A.conf -I /home/aconway/dispatch/python ==30894== ==30894== ==30894== HEAP SUMMARY: ==30894== in use at exit: 5,287,521 bytes in 30,091 blocks ==30894== total heap usage: 217,689 allocs, 187,598 frees, 31,096,921 bytes allocated ==30894== ==30894== 4,632 (2,368 direct, 2,264 indirect) bytes in 1 blocks are definitely lost in loss record 3,400 of 3,534 ==30894== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30894== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30894== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30894== by 0x4E83588: qd_server_connection (server.c:500) ==30894== by 0x4E84F4C: on_accept (server.c:531) ==30894== by 0x4E84F4C: handle_listener (server.c:698) ==30894== by 0x4E84F4C: handle (server.c:842) ==30894== by 0x4E85AAF: thread_run (server.c:919) ==30894== by 0x54EC36C: start_thread (pthread_create.c:456) ==30894== by 0x6064BBE: clone (clone.S:97) ==30894== ==30894== LEAK SUMMARY: ==30894== definitely lost: 2,368 bytes in 1 blocks ==30894== indirectly lost: 2,264 bytes in 30 blocks ==30894== possibly lost: 1,424,205 bytes in 5,779 blocks ==30894== still reachable: 3,858,684 bytes in 24,281 blocks ==30894== suppressed: 0 bytes in 0 blocks ==30894== Reachable blocks (those to which a pointer was found) are not shown. ==30894== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30894== ==30894== For counts of detected and suppressed errors, rerun with: -v ==30894== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) <<<< ---------------------------------------------------------------------- Ran 2 tests in 22.275s FAILED (errors=3) Start 19: system_tests_default_distribution 19/35 Test #19: system_tests_default_distribution .........***Failed 6.24 sec test_anonymous_sender (system_tests_default_distribution.DefaultDistributionTest) ... ok test_create_unavailable_receiver (system_tests_default_distribution.DefaultDistributionTest) ... ok test_create_unavailable_sender (system_tests_default_distribution.DefaultDistributionTest) ... ok test_general (system_tests_default_distribution.DefaultDistributionTest) ... ok ERROR ====================================================================== ERROR: tearDownClass (system_tests_default_distribution.DefaultDistributionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 30963 error: exit code 42, expected 0 qdrouterd -c test-router.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_default_distribution/DefaultDistributionTest/setUpClass/test-router-1.cmd >>>> ==30963== Memcheck, a memory error detector ==30963== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==30963== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==30963== Command: qdrouterd -c test-router.conf -I /home/aconway/dispatch/python ==30963== ==30963== ==30963== HEAP SUMMARY: ==30963== in use at exit: 5,095,353 bytes in 28,912 blocks ==30963== total heap usage: 182,373 allocs, 153,461 frees, 26,300,110 bytes allocated ==30963== ==30963== 5,312 (320 direct, 4,992 indirect) bytes in 1 blocks are definitely lost in loss record 3,348 of 3,464 ==30963== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==30963== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==30963== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==30963== by 0x4E7DEFD: qdr_link_deliver_to (transfer.c:71) ==30963== by 0x4E81589: AMQP_rx_handler (router_node.c:428) ==30963== by 0x4E6043E: do_receive (container.c:175) ==30963== by 0x4E6043E: qd_container_handle_event (container.c:523) ==30963== by 0x4E84DEC: handle (server.c:901) ==30963== by 0x4E85AAF: thread_run (server.c:919) ==30963== by 0x54EC36C: start_thread (pthread_create.c:456) ==30963== by 0x6064BBE: clone (clone.S:97) ==30963== ==30963== LEAK SUMMARY: ==30963== definitely lost: 320 bytes in 1 blocks ==30963== indirectly lost: 4,992 bytes in 14 blocks ==30963== possibly lost: 1,310,261 bytes in 5,167 blocks ==30963== still reachable: 3,779,780 bytes in 23,730 blocks ==30963== suppressed: 0 bytes in 0 blocks ==30963== Reachable blocks (those to which a pointer was found) are not shown. ==30963== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==30963== ==30963== For counts of detected and suppressed errors, rerun with: -v ==30963== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0) <<<< ---------------------------------------------------------------------- Ran 4 tests in 6.195s FAILED (errors=1) Start 20: system_tests_policy 20/35 Test #20: system_tests_policy .......................***Failed 18.67 sec test_verify_maximum_connections (system_tests_policy.AbsoluteConnectionCountLimit) ... ok test_repeated_create_delete (system_tests_policy.LoadPolicyFromFolder) ... ERROR test_verify_policies_are_loaded (system_tests_policy.LoadPolicyFromFolder) ... ERROR test_verify_policy_add_update_delete (system_tests_policy.LoadPolicyFromFolder) ... ERROR test_verify_n_receivers (system_tests_policy.SenderReceiverLimits) ... ok test_verify_n_senders (system_tests_policy.SenderReceiverLimits) ... ok ====================================================================== ERROR: test_repeated_create_delete (system_tests_policy.LoadPolicyFromFolder) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_policy.py", line 227, in test_repeated_create_delete rulesets = json.loads(self.run_qdmanage('query --type=vhost')) File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded ====================================================================== ERROR: test_verify_policies_are_loaded (system_tests_policy.LoadPolicyFromFolder) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_policy.py", line 124, in test_verify_policies_are_loaded rulesets = json.loads(self.run_qdmanage('query --type=vhost')) File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded ====================================================================== ERROR: test_verify_policy_add_update_delete (system_tests_policy.LoadPolicyFromFolder) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_tests_policy.py", line 183, in test_verify_policy_add_update_delete rulesets = json.loads(self.run_qdmanage('query --type=vhost')) File "/usr/lib64/python2.7/json/__init__.py", line 339, in loads return _default_decoder.decode(s) File "/usr/lib64/python2.7/json/decoder.py", line 364, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib64/python2.7/json/decoder.py", line 382, in raw_decode raise ValueError("No JSON object could be decoded") ValueError: No JSON object could be decoded ---------------------------------------------------------------------- Ran 6 tests in 18.626s FAILED (errors=3) Start 21: system_tests_protocol_family 21/35 Test #21: system_tests_protocol_family ..............***Failed 18.98 sec test_simple_pre_settled (system_tests_protocol_family.ProtocolFamilyTest) ... ok ERROR ====================================================================== ERROR: tearDownClass (system_tests_protocol_family.ProtocolFamilyTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/aconway/dispatch/tests/system_test.py", line 606, in tearDownClass cls.tester.teardown() File "/home/aconway/dispatch/tests/system_test.py", line 544, in teardown raise RuntimeError("Errors during teardown: \n\n%s" % "\n\n".join([str(e) for e in errors])) RuntimeError: Errors during teardown: Process 31068 error: exit code 42, expected 0 qdrouterd -c A.conf -I /home/aconway/dispatch/python /home/aconway/dispatch/reldbg/tests/system_test.dir/system_tests_protocol_family/ProtocolFamilyTest/setUpClass/A-1.cmd >>>> ==31068== Memcheck, a memory error detector ==31068== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==31068== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info ==31068== Command: qdrouterd -c A.conf -I /home/aconway/dispatch/python ==31068== ==31068== ==31068== HEAP SUMMARY: ==31068== in use at exit: 5,323,005 bytes in 30,225 blocks ==31068== total heap usage: 229,454 allocs, 199,229 frees, 33,338,922 bytes allocated ==31068== ==31068== 128 bytes in 2 blocks are definitely lost in loss record 2,038 of 3,532 ==31068== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==31068== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==31068== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==31068== by 0x4E605A5: setup_incoming_link (container.c:134) ==31068== by 0x4E605A5: qd_container_handle_event (container.c:455) ==31068== by 0x4E84DEC: handle (server.c:901) ==31068== by 0x4E85AAF: thread_run (server.c:919) ==31068== by 0x54EC36C: start_thread (pthread_create.c:456) ==31068== by 0x6064BBE: clone (clone.S:97) ==31068== ==31068== 4,632 (2,368 direct, 2,264 indirect) bytes in 1 blocks are definitely lost in loss record 3,397 of 3,532 ==31068== at 0x4C30D47: memalign (vg_replace_malloc.c:857) ==31068== by 0x4C30E45: posix_memalign (vg_replace_malloc.c:1020) ==31068== by 0x4E868D8: qd_alloc (alloc_pool.c:182) ==31068== by 0x4E83588: qd_server_connection (server.c:500) ==31068== by 0x4E84F4C: on_accept (server.c:531) ==31068== by 0x4E84F4C: handle_listener (server.c:698) ==31068== by 0x4E84F4C: handle (server.c:842) ==31068== by 0x4E85AAF: thread_run (server.c:919) ==31068== by 0x54EC36C: start_thread (pthread_create.c:456) ==31068== by 0x6064BBE: clone (clone.S:97) ==31068== ==31068== LEAK SUMMARY: ==31068== definitely lost: 2,496 bytes in 3 blocks ==31068== indirectly lost: 2,264 bytes in 30 blocks ==31068== possibly lost: 1,424,298 bytes in 5,765 blocks ==31068== still reachable: 3,893,947 bytes in 24,427 blocks ==31068== suppressed: 0 bytes in 0 blocks ==31068== Reachable blocks (those to which a pointer was found) are not shown. ==31068== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==31068== ==31068== For counts of detected and suppressed errors, rerun with: -v ==31068== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 0 from 0) <<<<