[THRIFT-2771] Dangling references / Leak when using hbase->mutateRows?

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9
• Component/s: PHP - Library
• Assignee: Unassigned
• Reporter: Sebastian Mach
• Created: 07/Oct/14 16:47
• Updated: 07/Oct/14 16:47

Description

When setting up our Hbase-Client like ...

    $socket = new TSocket( OI_HBASE_HOST, OI_HBASE_PORT );
    $socket->setSendTimeout( 30000 ); // Ten seconds (too long for production, but this is just a demo ;)
    $socket->setRecvTimeout( 60000 ); // Twenty seconds
    $this->transport = new TBufferedTransport( $socket );
    $protocol = new TBinaryProtocolAccelerated( $this->transport );
    $this->hbase = new HbaseClient( $protocol );

... and then using ...

    $this->hbase->mutateRows("user_crm", $batchMutations, null);

... multiple times on a longer running task, it seems that a memory leak of some sort is hit.

I tracked the memory-usage-growth to that call; before and after the call, measured with memory_get_usage(true) or memory_get_usage(false), the memory usage is identical, but when re-entering the paths multiple times, the total usage increases and increases.

I tried to explitly unset all variables, also within the function mutateRows, and invoked garbage collection explicitly with gc_collect_cycles();. I also tried lowering the maximum allowed memory for PHP to see if this will invoke the GC better, but then ran into "out of memory"-errors.

Within Thrift/packages/Hbase/Hbase.php::send_mutateRows, thrift_protocol_write_binary is called as intended.

I also checked https://issues.apache.org/jira/browse/THRIFT-1067, but we use version 0.9.0, where the patch is documented to be applied already.

The only clues I have at the moment:

• Before and after the call to mutateRows, memory usage is identical
• When I remove the call, there is no "leak", which hints (but surely doesn't prove) that our side is clean.
• Those clues combined with my own instinct, it looks like there's maybe a reference on the implementatory C-side that is dangling or held too long and can't be freed by the GC.

# thrift --version
Thrift version 0.9.0