[ZOOKEEPER-4699] zh->hostname heap-use-after-free in zookeeper_interest

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.8.1
• Fix Version/s: 3.10.0, 3.9.4
• Component/s: c client
• Labels: pull-request-available

Environment

debian 

• Assignee: fanyang
• Reporter: whyer
• Created: 24/May/23 20:12
• Updated: 28/Oct/24 05:38
• Resolved: 18/Oct/24 01:52

Description

we got an asan error. The usage is one separate thread call zoo_set_servers periodically. It will use lock to make (free and reset zh->hostname operation) atomic:
// NOTE: guard access to{hostname, addr_cur, addrs, addrs_old, addrs_new}lock_reconfig(zh);

in the mean while the io thread will call zoo_interest function and access zh->hostname in log: LOG_WARN(LOGCALLBACK(zh), "Delaying connection after exhaustively trying all servers [%s]",zh->hostname); without any lock...

 

 

stack:

{{=================================================================
==450==ERROR: AddressSanitizer: heap-use-after-free on address 0x6030004fcbd0 at pc 0x7fbc74e9a5ce bp 0x7fbc3ebf4060 sp 0x7fbc3ebf3810
READ of size 2 at 0x6030004fcbd0 thread T98
#0 0x7fbc74e9a5cd (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x8a5cd)
#1 0x7fbc74e9c61d in __interceptor_vsnprintf (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x8c61d)
#2 0x55e1ced0cdd6 in log_message (/opt/tiger/{}{}/deploy/bin/{}{}+0x1906dd6)
#3 0x55e1cecfc578 in zookeeper_interest (/opt/tiger/{}{}/deploy/bin/{}{}+0x18f6578)
#4 0x55e1ced0f0b4 in do_io (/opt/tiger/{}{}/deploy/bin/{}{}+0x19090b4)
#5 0x7fbc74bfa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3)
#6 0x7fbc73656d0e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0xe8d0e)

0x6030004fcbd0 is located 0 bytes inside of 20-byte region [0x6030004fcbd0,0x6030004fcbe4)
freed by thread T100 here:
#0 0x7fbc74ed1a10 in free (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1a10)
#1 0x55e1cecf9e14 in zoo_set_servers (/opt/tiger/{}{}/deploy/bin/{}{}+0x18f3e14)
#5 0x7fbc74bfa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3)

previously allocated by thread T100 here:
#0 0x7fbc74e67f30 in strdup (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x57f30)
#1 0x55e1cecf9e20 in zoo_set_servers (/opt/tiger/{}{}/deploy/bin/{}{}+0x18f3e20)
#5 0x7fbc74bfa4a3 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x74a3)

Thread T98 created by T0 here:
#0 0x7fbc74e40f59 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x30f59)
#1 0x55e1ced0ea97 in start_threads (/opt/tiger/{}{}/deploy/bin/{}{}+0x1908a97)
#2 0x55e1ced0ed11 in adaptor_init (/opt/tiger/{}{}/deploy/bin/{}{}+0x1908d11)
#3 0x55e1cecf9c9c in zookeeper_init_internal (/opt/tiger/{}{}/deploy/bin/{}{}+0x18f3c9c)
#4 0x55e1cecf9d38 in zookeeper_init (/opt/tiger/{}{}/deploy/bin/{}{}+0x18f3d38)
#12 0x55e1ce2f2097 in main /tmp/{}{}/{}{}/{}*/main.cc:148
#13 0x7fbc7358e2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)

Thread T100 created by T0 here:
#0 0x7fbc74e40f59 in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x30f59)
#8 0x55e1ce2f2097 in main /tmp/{}{}/{}{}/{}*/main.cc:148
#9 0x7fbc7358e2e0 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x202e0)

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x8a5cd)
Shadow bytes around the buggy address:
0x0c0680097920: 00 00 00 07 fa fa 00 00 00 07 fa fa fd fd fd fd
0x0c0680097930: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c0680097940: fa fa fa fa 00 00 00 07 fa fa 00 00 00 07 fa fa
0x0c0680097950: 00 00 00 07 fa fa fa fa fa fa fa fa 00 00 00 07
0x0c0680097960: fa fa fa fa fa fa fa fa fa fa fa fa fa fa 00 00
=>0x0c0680097970: 00 07 fa fa fa fa fa fa fa fa[fd]fd fd fa fa fa
0x0c0680097980: fd fd fd fa fa fa 00 00 00 07 fa fa fd fd fd fd
0x0c0680097990: fa fa fa fa fa fa fa fa 00 00 00 07 fa fa 00 00
0x0c06800979a0: 00 07 fa fa fd fd fd fd fa fa 00 00 00 07 fa fa
0x0c06800979b0: fa fa fa fa fa fa fa fa fa fa fa fa fd fd fd fd
0x0c06800979c0: fa fa fd fd fd fd fa fa fa fa fa fa fa fa fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb}}

 

 

Issue Links

• links to: GitHub Pull Request #2138