[DISPATCH-2056] AddressSanitizer: use-after-poison in qdr_connection_set_context during system_tests_tcp_adaptor, system_tests_http2

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Component/s: Protocol Adaptors
• Assignee: Unassigned
• Reporter: Jiri Daněk
• Created: 15/Apr/21 21:22
• Updated: 14/May/21 14:09
• Resolved: 10/May/21 15:01

Description

The pool poison PR is new and untried, so this report needs to be taken with a portion of healthy scepticism.

https://travis-ci.com/github/apache/qpid-dispatch/jobs/498888397#L30319

72: =================================================================
3216172: ==18570==ERROR: AddressSanitizer: use-after-poison on address 0x61800006fb18 at pc 0x7ffa2c7dab05 bp 0x7ffa226d1190 sp 0x7ffa226d1188
3216272: WRITE of size 8 at 0x61800006fb18 thread T4
3216372:     #0 0x7ffa2c7dab04 in qdr_connection_set_context /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:162:28
3216472:     #1 0x7ffa2c6de93c in handle_disconnected /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:364:9
3216572:     #2 0x7ffa2c6de93c in handle_connection_event /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:655:9
3216672:     #3 0x7ffa2c908291 in handle /home/travis/build/apache/qpid-dispatch/src/server.c
3216772:     #4 0x7ffa2c901c6f in thread_run /home/travis/build/apache/qpid-dispatch/src/server.c:1122:23
3216872:     #5 0x7ffa2c363608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
3216972:     #6 0x7ffa2bb8e292 in clone (/lib/x86_64-linux-gnu/libc.so.6+0x122292)
3217072: 
3217172: 0x61800006fb18 is located 664 bytes inside of 832-byte region [0x61800006f880,0x61800006fbc0)
3217272: allocated by thread T4 here:
3217372:     #0 0x496f97 in posix_memalign (/home/travis/build/apache/qpid-dispatch/build/router/qdrouterd+0x496f97)
3217472:     #1 0x7ffa2c6eff9e in qd_alloc /home/travis/build/apache/qpid-dispatch/src/alloc_pool.c:398:13
3217572:     #2 0x7ffa2c7d4c8e in new_qdr_connection_t /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:44:1
3217672:     #3 0x7ffa2c7d4c8e in qdr_connection_opened /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:89:32
3217772:     #4 0x7ffa2c6e16f7 in qdr_tcp_open_server_side_connection /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:761:30
3217872:     #5 0x7ffa2c6df1c0 in handle_connection_event /home/travis/build/apache/qpid-dispatch/src/adaptors/tcp_adaptor.c:625:17
3217972:     #6 0x7ffa2c908291 in handle /home/travis/build/apache/qpid-dispatch/src/server.c
3218072:     #7 0x7ffa2c901c6f in thread_run /home/travis/build/apache/qpid-dispatch/src/server.c:1122:23
3218172:     #8 0x7ffa2c363608 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x9608)
3218272: 
3218372: Thread T4 created by T0 here:
3218472:     #0 0x480f0a in pthread_create (/home/travis/build/apache/qpid-dispatch/build/router/qdrouterd+0x480f0a)
3218572:     #1 0x7ffa2c7a7b9d in sys_thread /home/travis/build/apache/qpid-dispatch/src/posix/threading.c:183:5
3218672:     #2 0x7ffa2c90152e in qd_server_run /home/travis/build/apache/qpid-dispatch/src/server.c:1485:22
3218772:     #3 0x4c7bbb in main_process /home/travis/build/apache/qpid-dispatch/router/src/main.c:115:5
3218872:     #4 0x4c6876 in main /home/travis/build/apache/qpid-dispatch/router/src/main.c:369:9
3218972:     #5 0x7ffa2ba930b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
3219072: 
3219172: SUMMARY: AddressSanitizer: use-after-poison /home/travis/build/apache/qpid-dispatch/src/router_core/connections.c:162:28 in qdr_connection_set_context
3219272: Shadow bytes around the buggy address:
3219372:   0x0c3080005f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3219472:   0x0c3080005f20: 00 00 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3219572:   0x0c3080005f30: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3219672:   0x0c3080005f40: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3219772:   0x0c3080005f50: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3219872: =>0x0c3080005f60: f7 f7 f7[f7]f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3219972:   0x0c3080005f70: f7 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
3220072:   0x0c3080005f80: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
3220172:   0x0c3080005f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
3220272:   0x0c3080005fa0: 00 00 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3220372:   0x0c3080005fb0: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7
3220472: Shadow byte legend (one shadow byte represents 8 application bytes):
3220572:   Addressable:           00
3220672:   Partially addressable: 01 02 03 04 05 06 07 
3220772:   Heap left redzone:       fa
3220872:   Freed heap region:       fd
3220972:   Stack left redzone:      f1
3221072:   Stack mid redzone:       f2
3221172:   Stack right redzone:     f3
3221272:   Stack after return:      f5
3221372:   Stack use after scope:   f8
3221472:   Global redzone:          f9
3221572:   Global init order:       f6
3221672:   Poisoned by user:        f7
3221772:   Container overflow:      fc
3221872:   Array cookie:            ac
3221972:   Intra object redzone:    bb
3222072:   ASan internal:           fe
3222172:   Left alloca redzone:     ca
3222272:   Right alloca redzone:    cb
3222372:   Shadow gap:              cc
3222472: ==18570==ABORTING

Issue Links

• is caused by: DISPATCH-2039 Memory pool should be manually poisoned so that ASAN works with it
• Testing discovered: DISPATCH-2109 Shutdown crash: member access within null pointer of type 'struct qdr_node_t'