[GEODE-1678] A GatewaySenderEventImpl which has reference to offheap memory may not be put into gateway region queue and cause offheap memory leak

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-incubating
• Component/s: offheap, wan
• Assignee: Eric Shu
• Reporter: Eric Shu
• Created: 19/Jul/16 22:38
• Updated: 25/Oct/16 07:16
• Resolved: 29/Jul/16 17:47

Description

The following stack trace shows when the offheap memory was retained but not released afterwards.

[info 2016/07/18 10:49:43.662 PDT <Distributed system shutdown hook> tid=0x18] orphaned @7f16ba2040a0 rc=1 history=[USED rc=5 useCount=1 by ServerConnection on port 23594 Thread 1 owner=com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventImpl@1901332338:
        at com.gemstone.gemfire.internal.cache.EntryEventImpl.convertToStoredObject(EntryEventImpl.java:1300)
        at com.gemstone.gemfire.internal.cache.EntryEventImpl.getOffHeapNewValue(EntryEventImpl.java:1283)
        at com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventImpl.initializeValue(GatewaySenderEventImpl.java:945)
        at com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventImpl.<init>(GatewaySenderEventImpl.java:318)
        at com.gemstone.gemfire.internal.cache.wan.GatewaySenderEventImpl.<init>(GatewaySenderEventImpl.java:253)
        at com.gemstone.gemfire.internal.cache.wan.serial.SerialGatewaySenderEventProcessor.enqueueEvent(SerialGatewaySenderEventProcessor.java:433)
        at com.gemstone.gemfire.internal.cache.wan.serial.ConcurrentSerialGatewaySenderEventProcessor.enqueueEvent(ConcurrentSerialGatewaySenderEventProcessor.java:154)
        at com.gemstone.gemfire.internal.cache.wan.serial.ConcurrentSerialGatewaySenderEventProcessor.enqueueEvent(ConcurrentSerialGatewaySenderEventProcessor.java:121)
        at com.gemstone.gemfire.internal.cache.wan.AbstractGatewaySender.distribute(AbstractGatewaySender.java:962)
        at com.gemstone.gemfire.internal.cache.LocalRegion.notifyGatewaySender(LocalRegion.java:6490)
        at com.gemstone.gemfire.internal.cache.BucketRegion.notifyGatewaySender(BucketRegion.java:670)
        at com.gemstone.gemfire.internal.cache.LocalRegion.basicPutPart2(LocalRegion.java:6026)
        at com.gemstone.gemfire.internal.cache.BucketRegion.basicPutPart2(BucketRegion.java:660)
        at com.gemstone.gemfire.internal.cache.AbstractRegionMap.basicPut(AbstractRegionMap.java:2737)
        at com.gemstone.gemfire.internal.cache.BucketRegion.virtualPut(BucketRegion.java:501)
        at com.gemstone.gemfire.internal.cache.PartitionedRegionDataStore.putLocally(PartitionedRegionDataStore.java:1275)
        at com.gemstone.gemfire.internal.cache.PartitionedRegionDataStore.putLocally(PartitionedRegionDataStore.java:1250)
        at com.gemstone.gemfire.internal.cache.PartitionedRegionDataView.putEntryOnRemote(PartitionedRegionDataView.java:107)
        at com.gemstone.gemfire.internal.cache.partitioned.PutAllPRMessage.doLocalPutAll(PutAllPRMessage.java:485)
        at com.gemstone.gemfire.internal.cache.PartitionedRegion.tryToSendOnePutAllMessage(PartitionedRegion.java:2552)
        at com.gemstone.gemfire.internal.cache.PartitionedRegion.sendMsgByBucket(PartitionedRegion.java:2343)
        at com.gemstone.gemfire.internal.cache.PartitionedRegion.postPutAllSend(PartitionedRegion.java:2109)
        at com.gemstone.gemfire.internal.cache.LocalRegionDataView.postPutAll(LocalRegionDataView.java:274)
        at com.gemstone.gemfire.internal.cache.LocalRegion.basicPutAll(LocalRegion.java:10203)
        at com.gemstone.gemfire.internal.cache.LocalRegion.basicBridgePutAll(LocalRegion.java:9892)
        at com.gemstone.gemfire.internal.cache.tier.sockets.command.PutAll80.cmdExecute(PutAll80.java:306)
        at com.gemstone.gemfire.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:146)
        at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doNormalMsg(ServerConnection.java:783)
        at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.doOneMessage(ServerConnection.java:913)
        at com.gemstone.gemfire.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1143)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at com.gemstone.gemfire.internal.cache.tier.sockets.AcceptorImpl$1$1.run(AcceptorImpl.java:555)
        at java.lang.Thread.run(Thread.java:745)