IMPALA-3203: Part 2: per-core free lists in buffer pool
Add per-core lists of clean pages and free pages to enable allocation
of buffers without contention on shared locks in the common case.
This is implemented with an additional layer of abstraction in
"BufferAllocator", which tracks all memory (free buffers and clean
pages) that is not in use but has not been released to the OS.
The old BufferAllocator is renamed to SystemAllocator.
See "Spilled Page Mgmt" and "MMap Allocator & Scalable Free Lists" in
https://goo.gl/0zuy97 for a high-level summary of how this fits into
the buffer pool design.
The guts of the new code is BufferAllocator::AllocateInternal(),
which progresses through several strategies for allocating memory.
- Enforce upper limit on buffer size to reduce the number of free lists
- Add additional allocation counters.
- Slightly reorganise the MemTracker GC functions to use lambdas and
clarify the order in which they should be called. Also adds a target
memory value so that they don't need to free all of the memory in
- Fix an accounting bug in the buffer pool where it didn't
evict dirty pages before reclaiming a clean page.
We will need to validate the performance of the system under high query
concurrency before this is used as part of query execution. The benchmark
in Part 1 provided some evidence that this approach of a list per core
should scale well to many cores.
Added buffer-allocator-test to test the free list resizing algorithm
Added a test to buffer-pool-test to exercise the various new memory
reclamation code paths that are now possible. Also run buffer-pool-test
under two different faked-out NUMA setups - one with no NUMA and another
with three NUMA nodes.
buffer-pool-test, suballocator-test, and buffered-tuple-stream-v2-test
provide some further basic coverage. Future system and unit tests will
validate this further before it is used for query execution (see
Ran an initial version of
IMPALA-4114, the ported BufferedBlockMgr
tests, against this. The randomised stress test revealed some accounting
bugs which are fixed. I'll post those tests as a follow-on patch.
Reviewed-by: Tim Armstrong <email@example.com>
Tested-by: Impala Public Jenkins