[SOLR-8688] Concurrent[LRU|LFU]Cache should clear() upon destroy() to reduce GC stress

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 6.0
• Component/s: search
• Assignee: Unassigned
• Reporter: Steve Shabino
• Created: 17/Feb/16 20:32
• Updated: 09/May/16 18:45

Description

When a SolrIndexSearcher is close()'d, it calls close() on all of its caches. FastLRUCache and LFUCache then call Concurrent[LRU|LFU]Cache's destroy(). The destroy method stops the clean-up thread if present, but it does not evict all cache entries - no longer of any value to a destroyed cache.

Because Concurrent[LRU|LFU]Cache has a finalize() method, it and all objects referenced by it, even indirectly, may not be GC'ed until the JVM performs a major GC and the finalization thread runs.

Calling clear() on the underlying ConcurrentHashMap after stopping the clean-up thread will free cache entries (Several gigs worth as we're currently configured in production) for GC, independent of the JVM's finalization process.

Alternatively, uses of the finalize() method could be replaced with a clean-up scheme which makes use of PhantomReferences. There are a few other uses of finalize() in Solr which also could benefit from this. An example project (that I've only perused briefly): https://github.com/claudemartin/java-cleanup

I would be happy to prepare patches for either of these schemes.