[ARROW-8861] [C++][Plasma] Memory not released until Plasma process is killed

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.16.0
• Component/s: C++ - Plasma
• Environment: Singularity container (Ubuntu 18.04)
• External issue URL: https://github.com/apache/arrow/issues/17190
• Assignee: Unassigned
• Reporter: Chengxin Ma
• Created: 19/May/20 15:54
• Updated: 11/Jan/23 08:03

Description

Invoking the Delete(const ObjectID& object_id) method of a plasma client seems not really to free up the memory used by the object.

To reproduce:
1. use htop (or other similar tools) to monitor memory usage;
2. start up the Plasma Object Store by plasma_store -m 1000000000 -s /tmp/plasma;
3. use put.py to put an object into Plasma;
4. compile and run delete.cc (g++ delete.cc `pkg-config --cflags --libs arrow plasma` --std=c++11 -o delete);
5. kill the plasma_store process.

Memory usage drops at Step 5, rather than Step 4.

How to free up the memory while keeping Plasma Object Store running?

put.py:

from pyarrow import plasma

if __name__ == "__main__":
    client = plasma.connect("/tmp/plasma")
    object_id = plasma.ObjectID(20 * b"a")
    object_size = 500000000
    buffer = memoryview(client.create(object_id, object_size))
    for i in range(500000000):
        buffer[i] = i % 128
    client.seal(object_id)
    client.disconnect()

delete.cc:

#include "arrow/util/logging.h"
#include <plasma/client.h>

using namespace plasma;

int main(int argc, char **argv)
{
    PlasmaClient client;
    ARROW_CHECK_OK(client.Connect("/tmp/plasma"));
    ObjectID object_id = ObjectID::from_binary("aaaaaaaaaaaaaaaaaaaa");

    client.Delete(object_id);

    ARROW_CHECK_OK(client.Disconnect());
}