[HDDS-1615] ManagedChannel references are being leaked in ReplicationSupervisor.java

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.5.0
• Component/s: Ozone Datanode
• Labels: MiniOzoneChaosCluster pull-request-available
• Target Version/s: 0.5.0
• Assignee: Mukul Kumar Singh Assign to me
• Reporter: Mukul Kumar Singh
• Created: 30/May/19 10:07
• Updated: 08/Mar/23 21:12
• Resolved: 01/Oct/19 09:02

Description

ManagedChannel references are being leaked in ReplicationSupervisor.java

May 30, 2019 8:10:56 AM org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference cleanQueue
SEVERE: *~*~*~ Channel ManagedChannelImpl{logId=1495, target=192.168.0.3:49868} was not shutdown properly!!! ~*~*~*
    Make sure to call shutdown()/shutdownNow() and wait until awaitTermination() returns true.
java.lang.RuntimeException: ManagedChannel allocation site
        at org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper$ManagedChannelReference.<init>(ManagedChannelOrphanWrapper.java:103)
        at org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:53)
        at org.apache.ratis.thirdparty.io.grpc.internal.ManagedChannelOrphanWrapper.<init>(ManagedChannelOrphanWrapper.java:44)
        at org.apache.ratis.thirdparty.io.grpc.internal.AbstractManagedChannelImplBuilder.build(AbstractManagedChannelImplBuilder.java:411)
        at org.apache.hadoop.ozone.container.replication.GrpcReplicationClient.<init>(GrpcReplicationClient.java:65)
        at org.apache.hadoop.ozone.container.replication.SimpleContainerDownloader.getContainerDataFromReplicas(SimpleContainerDownloader.java:87)
        at org.apache.hadoop.ozone.container.replication.DownloadAndImportReplicator.replicate(DownloadAndImportReplicator.java:118)
        at org.apache.hadoop.ozone.container.replication.ReplicationSupervisor$TaskRunner.run(ReplicationSupervisor.java:115)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

Issue Links

• relates to: HDDS-8108 gRPC channel created for replication client not shutdown properly
• links to: GitHub Pull Request #1547