[HBASE-16054] OutOfMemory exception when using AsyncRpcClient with encryption

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0, 2.0.0
• Hadoop Flags: Reviewed
• Assignee: Colin
• Reporter: Colin
• Created: 17/Jun/16 06:42
• Updated: 20/Jun/16 19:19
• Resolved: 20/Jun/16 14:32

Description

Test the AsyncRpcClient with encryption in infinity loop, will get the OOM exception like the following:

io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 29106160 byte(s) of direct memory (used: 2607219760, max: 2609905664)
at io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:592)
at io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:546)
at io.netty.buffer.PoolArena$DirectArena.allocateDirect(PoolArena.java:699)
at io.netty.buffer.PoolArena$DirectArena.newUnpooledChunk(PoolArena.java:695)
at io.netty.buffer.PoolArena.allocateHuge(PoolArena.java:246)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:224)
at io.netty.buffer.PoolArena.allocate(PoolArena.java:141)
at io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:262)
at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
at io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:170)
at io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:107)
at org.apache.hadoop.hbase.security.SaslClientHandler.write(SaslClientHandler.java:328)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:724)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:716)
at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:36)
at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1064)
at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1111)
at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1049)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:339)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:393)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:742)
at java.lang.Thread.run(Thread.java:745)