This is being split out from
IMPALA-7446, since it's a somewhat separate issue.
The issue is that the process memory consumption only accounts for system memory allocated from the buffer pool, not for reservations that have not been fulfilled yet. Reserved allocations do not check the process memory limit before going through, so it's possible that the same memory could be committed both to a non-reserved allocation and a reserved allocation.
In practice this seems unlikely to cause any real symptoms - it would require:
- the process to be near its memory limit
- a large amount of unused reservation, not offset by free buffers or clean pages.
- a quick spike in the amount of reservation used
This could result in the process going significantly over its memory limit before queries start to fail and free memory.