Uploaded image for project: 'Qpid Proton'
  1. Qpid Proton
  2. PROTON-2436

TSAN race in epoll.c post_event with raw connection

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • proton-c-0.36.0
    • proton-c-0.37.0
    • proton-c
    • None

    Description

      today's github CI run of dispatch+proton main kicked up a tsan error in proton I've never seen before:

      https://github.com/apache/qpid-dispatch/runs/3700836319?check_suite_focus=true#step:27:2142
       

      70: WARNING: ThreadSanitizer: data race (pid=3075)
70:   Write of size 4 at 0x7b680000dd38 by main thread (mutexes: write M257):
70:     #0 post_event /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2304 (libqpid-proton-proactor.so.1+0x14108)
70:     #1 poller_do_epoll /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2534 (libqpid-proton-proactor.so.1+0x14108)
70:     #2 next_event_batch /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2438 (libqpid-proton-proactor.so.1+0x14108)
70:     #3 pn_proactor_wait /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2650 (libqpid-proton-proactor.so.1+0x14622)
70:     #4 thread_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1118 (qdrouterd+0x4d83a9)
70:     #5 qd_server_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1527 (qdrouterd+0x4d904c)
70:     #6 main_process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:115 (qdrouterd+0x426cdc)
70:     #7 main /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x42623c)
70: 
70:   Previous read of size 4 at 0x7b680000dd38 by thread T3 (mutexes: write M499):
70:     #0 pni_raw_connection_process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll_raw_connection.c:355 (libqpid-proton-proactor.so.1+0x108ec)
70:     #1 process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2230 (libqpid-proton-proactor.so.1+0x108ec)
70:     #2 next_event_batch /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2419 (libqpid-proton-proactor.so.1+0x108ec)
70:     #3 pn_proactor_wait /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2650 (libqpid-proton-proactor.so.1+0x14622)
70:     #4 thread_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1118 (qdrouterd+0x4d83a9)
70:     #5 _thread_init /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/posix/threading.c:172 (qdrouterd+0x47fe2d)
70: 
70:   Location is heap block of size 1536 at 0x7b680000d800 allocated by main thread:
70:     #0 calloc <null> (libtsan.so.0+0x32b3e)
70:     #1 pn_raw_connection /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll_raw_connection.c:168 (libqpid-proton-proactor.so.1+0xdf82)
70:     #2 _do_reconnect /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/adaptors/http1/http1_server.c:451 (qdrouterd+0x43da47)
70:     #3 qd_timer_visit /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/timer.c:316 (qdrouterd+0x4daddf)
70:     #4 handle /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1018 (qdrouterd+0x4d60d6)
70:     #5 thread_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1133 (qdrouterd+0x4d84e7)
70:     #6 qd_server_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1527 (qdrouterd+0x4d904c)
70:     #7 main_process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:115 (qdrouterd+0x426cdc)
70:     #8 main /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x42623c)
70: 
70:   Mutex M257 (0x7b640003aa20) created at:
70:     #0 pthread_mutex_init <null> (libtsan.so.0+0x49603)
70:     #1 pmutex_init /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll-internal.h:323 (libqpid-proton-proactor.so.1+0xd52c)
70:     #2 pn_proactor /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:1941 (libqpid-proton-proactor.so.1+0xd52c)
70:     #3 qd_server /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1380 (qdrouterd+0x4d872f)
70:     #4 qd_dispatch_prepare /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/dispatch.c:334 (qdrouterd+0x4604b6)
70:     #5 ffi_call_unix64 <null> (libffi.so.6+0x6c03)
70:     #6 main_process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:97 (qdrouterd+0x426c7c)
70:     #7 main /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x42623c)
70: 
70:   Mutex M499 (0x7b680000d800) created at:
70:     #0 pthread_mutex_init <null> (libtsan.so.0+0x49603)
70:     #1 pmutex_init /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll-internal.h:323 (libqpid-proton-proactor.so.1+0x146eb)
70:     #2 task_init /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:224 (libqpid-proton-proactor.so.1+0x146eb)
70:     #3 praw_connection_init /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll_raw_connection.c:141 (libqpid-proton-proactor.so.1+0x146eb)
70:     #4 pn_proactor_raw_connect /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll_raw_connection.c:179 (libqpid-proton-proactor.so.1+0x146eb)
70:     #5 _do_reconnect /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/adaptors/http1/http1_server.c:455 (qdrouterd+0x43daa2)
70:     #6 qd_timer_visit /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/timer.c:316 (qdrouterd+0x4daddf)
70:     #7 handle /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1018 (qdrouterd+0x4d60d6)
70:     #8 thread_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1133 (qdrouterd+0x4d84e7)
70:     #9 qd_server_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1527 (qdrouterd+0x4d904c)
70:     #10 main_process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:115 (qdrouterd+0x426cdc)
70:     #11 main /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x42623c)
70: 
70:   Thread T3 (tid=3079, running) created by main thread at:
70:     #0 pthread_create <null> (libtsan.so.0+0x5bf45)
70:     #1 sys_thread /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/posix/threading.c:181 (qdrouterd+0x4802bc)
70:     #2 qd_server_run /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/src/server.c:1525 (qdrouterd+0x4d902c)
70:     #3 main_process /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:115 (qdrouterd+0x426cdc)
70:     #4 main /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-dispatch/router/src/main.c:369 (qdrouterd+0x42623c)
70: 
70: SUMMARY: ThreadSanitizer: data race /home/runner/work/qpid-dispatch/qpid-dispatch/qpid-proton/c/src/proactor/epoll.c:2304 in post_event
70: ==================
70: ThreadSanitizer: reported 1 warnings

       
       
      Proton main latest (commit 30b245a48d7461fd41854e5bb63eadf19cbb26bd)
       

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. DISPATCH-2181 Data race coming from Proton (pni_raw_connection_process/post_event) in system_tests_tcp_adaptor

          • Major - Major loss of function.
          • Closed
          relates to

          Bug - A problem which impairs or prevents the functions of the product. PROTON-2472 assert epoll.c:2519: poller_do_epoll: Assertion `!p->sched_ready_first' failed.

          • Major - Major loss of function.
          • Closed

          Activity

            People

              cliffjansen Clifford Jansen
              kgiusti Ken Giusti
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: