Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22963

Netty ByteBuf leak in rpc client implementation

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.3.0, 2.1.7, 2.2.2
    • Component/s: rpc
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      2019-08-31 21:48:27,067 ERROR [RS-EventLoopGroup-6-3] util.ResourceLeakDetector(317): LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
      Recent access records: 
      Created at:
      	org.apache.hbase.thirdparty.io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:96)
      	org.apache.hbase.thirdparty.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:187)
      	org.apache.hbase.thirdparty.io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:178)
      	org.apache.hbase.thirdparty.io.netty.buffer.Unpooled.directBuffer(Unpooled.java:125)
      	org.apache.hadoop.hbase.ipc.NettyRpcConnection.<init>(NettyRpcConnection.java:96)
      	org.apache.hadoop.hbase.ipc.NettyRpcClient.createConnection(NettyRpcClient.java:74)
      	org.apache.hadoop.hbase.ipc.NettyRpcClient.createConnection(NettyRpcClient.java:40)
      	org.apache.hadoop.hbase.ipc.AbstractRpcClient.getConnection(AbstractRpcClient.java:364)
      	org.apache.hadoop.hbase.ipc.AbstractRpcClient.callMethod(AbstractRpcClient.java:433)
      	org.apache.hadoop.hbase.ipc.AbstractRpcClient.access$300(AbstractRpcClient.java:97)
      	org.apache.hadoop.hbase.ipc.AbstractRpcClient$RpcChannelImplementation.callMethod(AbstractRpcClient.java:605)
      	org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$Stub.scan(ClientProtos.java:42345)
      	org.apache.hadoop.hbase.client.AsyncClientScanner.callOpenScanner(AsyncClientScanner.java:152)
      	org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.call(AsyncSingleRequestRpcRetryingCaller.java:82)
      	org.apache.hadoop.hbase.client.AsyncSingleRequestRpcRetryingCaller.lambda$doCall$7(AsyncSingleRequestRpcRetryingCaller.java:115)
      	org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
      	java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      	java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      	org.apache.hadoop.hbase.client.AsyncRegionLocator.lambda$getRegionLocation$3(AsyncRegionLocator.java:124)
      	org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
      	java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      	java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      	org.apache.hadoop.hbase.client.ConnectionUtils.lambda$getOrFetch$6(ConnectionUtils.java:573)
      	org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
      	java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      	java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      	org.apache.hadoop.hbase.client.ZKAsyncRegistry.tryComplete(ZKAsyncRegistry.java:123)
      	org.apache.hadoop.hbase.client.ZKAsyncRegistry.lambda$getMetaRegionLocation$1(ZKAsyncRegistry.java:165)
      	org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
      	java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      	java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      	org.apache.hadoop.hbase.client.ZKAsyncRegistry.lambda$getAndConvert$0(ZKAsyncRegistry.java:81)
      	org.apache.hadoop.hbase.util.FutureUtils.lambda$addListener$0(FutureUtils.java:68)
      	java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
      	java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
      	java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      	java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      	org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient$ZKTask$1.exec(ReadOnlyZKClient.java:174)
      	org.apache.hadoop.hbase.zookeeper.ReadOnlyZKClient.run(ReadOnlyZKClient.java:342)
      	java.lang.Thread.run(Thread.java:748)
      

      https://builds.apache.org/job/HBase-Flaky-Tests/job/master/4148/testReport/junit/org.apache.hadoop.hbase.client/TestConnection/testConnectionCloseAllowsInterrupt/

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zhangduo Duo Zhang
                Reporter:
                zhangduo Duo Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: