[DISPATCH-2039] Memory pool should be manually poisoned so that ASAN works with it

• Type: Wish
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.17.0
• Assignee: Jiri Daněk
• Reporter: Jiri Daněk
• Created: 06/Apr/21 21:05
• Updated: 23/Aug/21 15:01
• Resolved: 07/May/21 15:09

Description

From https://github.com/google/sanitizers/wiki/AddressSanitizerManualPoisoning

A user may poison/unpoison a region of memory manually. Use this feature with caution. In many cases good old malloc+free is a better way to find heap bugs than using custom allocators with manual poisoning.

As far as I can tell, it is nowadays not possible to turn off the pool allocation and use malloc/free, because the pool mechanism also implements the weak pointers and ref counters. That means giving hints to ASAN is the only way to discover memory bugs of the type (if what Chuck speculated is true) of DISPATCH-2032.

If you have a custom allocation arena, the typical workflow would be to poison the entire arena first, and then unpoison allocated chunks of memory leaving poisoned redzones between them. The allocated chunks should start with 8-aligned addresses.

Alternatively, the current memory debugging machinery for the pool could take care of it on its own... but using ASAN seems sensible to me.

http://blog.hostilefork.com/poison-memory-without-asan/

Nice to have extra features (which won't be implemented at first)

• redzones, there should be chunks of poison on either end of a returned memory, to detect invalid accesses out of bounds; this means deliberate waste of memory (I am thinking 3x increase, to make implementation easy)
• quarantine, returned chunks should be kept in the pool for some time before they are returned as new allocations, to catch use-after-free; this policy goes against performance considerations

Open issues

Is it necessary to lock around the poison macros? I did not understand the thread safety note in API comment fully.

One thought

Actually, setting a limit on free_list length == 0 would effectively disable pool and turn the calls into simple wrappers over malloc/free. It would be enough to make this configurable at build time. Then asan should work just fine without poison.

Issue Links

• causes: DISPATCH-2076 [ASan] use-after-poison in qd_connector_decref during system_tests_edge_router
• causes: DISPATCH-2056 AddressSanitizer: use-after-poison in qdr_connection_set_context during system_tests_tcp_adaptor, system_tests_http2
• causes: DISPATCH-2095 heap-use-after-free in system_tests_policy_oversize_compound
• causes: DISPATCH-2045 qd_hash_internal_remove_item writes to freed (pooled) memory on router shutdown
• is related to: DISPATCH-2060 use-after free in qd_alloc_deref_safe_ptr if a pool item has been freed due to global_free_list size limit