Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
6.5
-
None
-
None
Description
As a follow up from SOLR-10506 we found another possible memory leak in Solr. The values generated from an ExternalFileField are cached in a static cache inside the FileFloatSource. That cache caches both a IndexReader and FileFloatSource}}s loaded using that {{IndexReader.
Cache eviction is left to the internally used WeakHashMap or a full eviction can be triggered via url. We are dealing with large synonym files and word lists stored in managed resources. Those are tied to the SolrCore as described in SOLR-10506. We're also using ExternalFileField}}s whose {{FileFloatSource are cached in said static cache. The FileFloatSource hold strong (transitive) references to the SolrCore they have been created for.
After a couple of collection reloads, the cache eviction mechanism of the WeakHashMap gets activated pretty close to heap exhaustion. The patch attached adds a mechanism to evict cache entries created in the context of a SolrCore upon it's close using a close hook in the ExternalFileFieldReloader. It furthermore adds a static cache reset method for all entries bound to a given IndexReader. I'm not sure, if the added cache resets are too aggressive or executed too often, I'd like to leave that to the experts 
