[DISPATCH-2128] delivery_tracker core process walks link->undelivered unlocked

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.0
• Fix Version/s: 1.20.0
• Component/s: Router Node
• Labels: race-condition tsan
• Assignee: Ken Giusti
• Reporter: Ken Giusti
• Created: 17/May/21 21:38
• Updated: 15/Mar/22 21:54

Description

While core thread walks the qdr_link_t->undelivered list, the I/O thread can modify it:

 

5: WARNING: ThreadSanitizer: data race (pid=569945)
65: Write of size 8 at 0x7b5800066158 by thread T3 (mutexes: write M380):
65: #0 qdr_link_process_deliveries /home/kgiusti/work/dispatch/qpid-dispatch/src/router_core/transfer.c:212 (libqpid-dispatch.so+0x104875)
65: #1 CORE_link_push /home/kgiusti/work/dispatch/qpid-dispatch/src/router_node.c:1920 (libqpid-dispatch.so+0x127d00)
65: #2 qdr_connection_process /home/kgiusti/work/dispatch/qpid-dispatch/src/router_core/connections.c:414 (libqpid-dispatch.so+0xc4bec)
65: #3 AMQP_writable_conn_handler /home/kgiusti/work/dispatch/qpid-dispatch/src/router_node.c:299 (libqpid-dispatch.so+0x122d42)
65: #4 writable_handler /home/kgiusti/work/dispatch/qpid-dispatch/src/container.c:395 (libqpid-dispatch.so+0x7b2e2)
65: #5 qd_container_handle_event /home/kgiusti/work/dispatch/qpid-dispatch/src/container.c:747 (libqpid-dispatch.so+0x7cfd5)
65: #6 handle /home/kgiusti/work/dispatch/qpid-dispatch/src/server.c:1096 (libqpid-dispatch.so+0x130537)
65: #7 thread_run /home/kgiusti/work/dispatch/qpid-dispatch/src/server.c:1121 (libqpid-dispatch.so+0x13063a)
65: #8 _thread_init /home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:172 (libqpid-dispatch.so+0xad37a)
65: #9 <null> <null> (libtsan.so.0+0x2d33f)
65:
65: Previous read of size 8 at 0x7b5800066158 by thread T1:
65: #0 process_link_CT /home/kgiusti/work/dispatch/qpid-dispatch/src/router_core/modules/stuck_delivery_detection/delivery_tracker.c:64 (libq\
pid-dispatch.so+0x11b745)
65: #1 action_handler_CT /home/kgiusti/work/dispatch/qpid-dispatch/src/router_core/modules/stuck_delivery_detection/delivery_tracker.c:115 (l\
ibqpid-dispatch.so+0x11bbe8)
65: #2 router_core_process_background_action_LH /home/kgiusti/work/dispatch/qpid-dispatch/src/router_core/router_core_thread.c:192 (libqpid-d\
ispatch.so+0xfadfd)
65: #3 router_core_thread /home/kgiusti/work/dispatch/qpid-dispatch/src/router_core/router_core_thread.c:220 (libqpid-dispatch.so+0xfaf4b)
65: #4 _thread_init /home/kgiusti/work/dispatch/qpid-dispatch/src/posix/threading.c:172 (libqpid-dispatch.so+0xad37a)
65: #5 <null> <null> (libtsan.so.0+0x2d33f)