We seem to be estimating the size of the on-heap memtable metadata to be around half of what it actually is. For example, during a read benchmark which writes 1 million single-long rows the memtable reports
while a heap dump taken at this point:
lists an usage of about 666MB altogether.
Switching to offheap_objects, the reported numbers are
while actual heap usage:
is about 442MB.
Looking at the code we definitely are not counting the AtomicBTreePartition.Holder, EncodingStats, liveness and deletion info objects associated with each partition, and most probably others.