[DISPATCH-2202] TSan data race from qdr_delivery_continue_peers_CT in system_tests_tcp_adaptor

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.17.0
• Labels: race-condition tsan
• Assignee: Unassigned
• Reporter: Jiri Daněk
• Created: 18/Jul/21 09:48
• Updated: 03/Sep/21 10:29

Description

https://app.travis-ci.com/github/apache/qpid-dispatch/jobs/525135183#L7676

73: WARNING: ThreadSanitizer: data race (pid=16033)
73:   Read of size 8 at 0x7b4400019d58 by thread T1 (mutexes: write M3385):
73:     #0 qdr_delivery_continue_peers_CT /home/travis/build/apache/qpid-dispatch/src/router_core/delivery.c:1055 (qdrouterd+0xa9150)
73:     #1 qdr_delivery_continue_CT /home/travis/build/apache/qpid-dispatch/src/router_core/delivery.c:1118 (qdrouterd+0xa966e)
73:     #2 router_core_thread /home/travis/build/apache/qpid-dispatch/src/router_core/router_core_thread.c:239 (qdrouterd+0xbf8ef)
73:     #3 _thread_init /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:172 (qdrouterd+0x8b3a1)
73: 
73:   Previous write of size 8 at 0x7b4400019d58 by main thread (mutexes: write M351):
73:     #0 qdr_link_process_deliveries /home/travis/build/apache/qpid-dispatch/src/router_core/transfer.c:237 (qdrouterd+0xc6426)
73:     #1 qdr_tcp_push /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:1519 (qdrouterd+0x4e9e2)
73:     #2 qdr_connection_process /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:414 (qdrouterd+0x9c7e2)
73:     #3 on_activate /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:196 (qdrouterd+0x4f761)
73:     #4 qd_timer_visit /home/travis/build/apache/qpid-dispatch/src/timer.c:315 (qdrouterd+0xe7902)
73:     #5 handle /home/travis/build/apache/qpid-dispatch/src/server.c:1018 (qdrouterd+0xe2ece)
73:     #6 thread_run /home/travis/build/apache/qpid-dispatch/src/server.c:1133 (qdrouterd+0xe4937)
73:     #7 qd_server_run /home/travis/build/apache/qpid-dispatch/src/server.c:1524 (qdrouterd+0xe54cf)
73:     #8 main_process /home/travis/build/apache/qpid-dispatch/router/src/main.c:115 (qdrouterd+0x2dc57)
73:     #9 main /home/travis/build/apache/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x2d0b2)
73: 
73:   Location is heap block of size 320 at 0x7b4400019c80 allocated by thread T2:
73:     #0 posix_memalign ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:835 (libtsan.so.0+0x3046d)
73:     #1 qd_alloc /home/travis/build/apache/qpid-dispatch/src/alloc_pool.c:396 (qdrouterd+0x540e4)
73:     #2 new_qdr_delivery_t /home/travis/build/apache/qpid-dispatch/src/router_core/delivery.c:24 (qdrouterd+0xa6ec2)
73:     #3 qdr_link_deliver /home/travis/build/apache/qpid-dispatch/src/router_core/transfer.c:47 (qdrouterd+0xc53ad)
73:     #4 AMQP_rx_handler /home/travis/build/apache/qpid-dispatch/src/router_node.c:870 (qdrouterd+0xdca91)
73:     #5 do_receive /home/travis/build/apache/qpid-dispatch/src/container.c:227 (qdrouterd+0x697d4)
73:     #6 qd_container_handle_event /home/travis/build/apache/qpid-dispatch/src/container.c:738 (qdrouterd+0x697d4)
73:     #7 handle /home/travis/build/apache/qpid-dispatch/src/server.c:1108 (qdrouterd+0xe2e01)
73:     #8 thread_run /home/travis/build/apache/qpid-dispatch/src/server.c:1133 (qdrouterd+0xe4838)
73:     #9 _thread_init /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:172 (qdrouterd+0x8b3a1)
73: 
73:   Mutex M3385 (0x7b1000054540) created at:
73:     #0 pthread_mutex_init ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1220 (libtsan.so.0+0x4a616)
73:     #1 sys_mutex /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:43 (qdrouterd+0x8b40f)
73:     #2 qdr_connection_opened /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:111 (qdrouterd+0x9b497)
73:     #3 qdr_tcp_open_server_side_connection /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:1003 (qdrouterd+0x4acee)
73:     #4 handle_connection_event /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:812 (qdrouterd+0x51ab8)
73:     #5 handle_event_with_context /home/travis/build/apache/qpid-dispatch/src/server.c:814 (qdrouterd+0xe2e59)
73:     #6 do_handle_raw_connection_event /home/travis/build/apache/qpid-dispatch/src/server.c:820 (qdrouterd+0xe2e59)
73:     #7 handle /home/travis/build/apache/qpid-dispatch/src/server.c:1101 (qdrouterd+0xe2e59)
73:     #8 thread_run /home/travis/build/apache/qpid-dispatch/src/server.c:1133 (qdrouterd+0xe4937)
73:     #9 _thread_init /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:172 (qdrouterd+0x8b3a1)
73: 
73:   Mutex M351 (0x7b100000e040) created at:
73:     #0 pthread_mutex_init ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1220 (libtsan.so.0+0x4a616)
73:     #1 sys_mutex /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:43 (qdrouterd+0x8b40f)
73:     #2 qdr_connection_opened /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:111 (qdrouterd+0x9b497)
73:     #3 qdr_tcp_open_server_side_connection /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:1003 (qdrouterd+0x4acee)
73:     #4 qdr_tcp_connection_egress /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:1089 (qdrouterd+0x4b756)
73:     #5 qd_dispatch_configure_tcp_connector /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:1327 (qdrouterd+0x53468)
73:     #6 <null> <null> (libffi.so.7+0x6ff4)
73:     #7 main_process /home/travis/build/apache/qpid-dispatch/router/src/main.c:97 (qdrouterd+0x2dbe2)
73:     #8 main /home/travis/build/apache/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x2d0b2)
73: 
73:   Thread T1 (tid=16035, running) created by main thread at:
73:     #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x5ea79)
73:     #1 sys_thread /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:181 (qdrouterd+0x8b962)
73:     #2 qdr_core /home/travis/build/apache/qpid-dispatch/src/router_core/router_core.c:124 (qdrouterd+0xb9bae)
73:     #3 qd_router_setup_late /home/travis/build/apache/qpid-dispatch/src/router_node.c:2120 (qdrouterd+0xdd94c)
73:     #4 <null> <null> (libffi.so.7+0x6ff4)
73:     #5 main_process /home/travis/build/apache/qpid-dispatch/router/src/main.c:97 (qdrouterd+0x2dbe2)
73:     #6 main /home/travis/build/apache/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x2d0b2)
73: 
73:   Thread T2 (tid=16036, running) created by main thread at:
73:     #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:962 (libtsan.so.0+0x5ea79)
73:     #1 sys_thread /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:181 (qdrouterd+0x8b962)
73:     #2 qd_server_run /home/travis/build/apache/qpid-dispatch/src/server.c:1522 (qdrouterd+0xe54aa)
73:     #3 main_process /home/travis/build/apache/qpid-dispatch/router/src/main.c:115 (qdrouterd+0x2dc57)
73:     #4 main /home/travis/build/apache/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x2d0b2)
73: 
73: SUMMARY: ThreadSanitizer: data race /home/travis/build/apache/qpid-dispatch/src/router_core/delivery.c:1055 in qdr_delivery_continue_peers_CT
73: ==================
73: ThreadSanitizer: reported 1 warnings
73: <<<<
73: 
73: ----------------------------------------------------------------------
73: Ran 11 tests in 31.982s
73: 
73: FAILED (errors=1, skipped=2)
73/75 Test #73: system_tests_tcp_adaptor ..........................***Failed   32.10 sec

Issue Links

• relates to: DISPATCH-2245 [TSan] data race in qdr_link_process_deliveries during system_tests_delivery_counts