Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
OutOfMemory Exception is observed when encountered with the issue IGNITE-8354. Though the issue is solved, preventing OOM by preventing unnecessary refresh of Cassandra session refresh, there seems to be a memory leak in ignite-cassandra module while using RoundRobinPolicy LoadBalancingPolicy while refreshing cassandra session. which seems to be the root cause of OOM.
In org.apache.ignite.cache.store.cassandra.session.CassandraSessionImpl.java
when refresh() method is invoked to handle Exceptions, new Cluster is build
with same LoadBalancingPolicy Object. We are using RoundRobinPolicy so same
RoundRobinPolicy object would be used while building Cluster when refresh()
is invoked. In RoundRobinPolicy there is a CopyOnWriteArrayList<Host>
liveHosts. When ever init(Cluster cluster, Collection<Host> hosts) is called
on RoundRobinPolicy it calls liveHosts.addAll(hosts) adding all the Host
Object Collection to liveHosts.
When ever Cluster is build during refresh() the Host Collection are added during init call,
again to the liveHosts of the same RoundRobinPolicy .Thus same Hosts are added again to liveHosts for every refresh() and the size would grow indefinitely after many refresh() calls causing OOM. Even in the heap dump post OOM we found huge number of Objects in liveHosts of RoundRobinPolicy Object.
Some possible solutions would be
1. To use new LoadBalancingPolicy object while building new Cluster during
refresh().
2. Somehow clear Objects in liveHosts during refresh().