[SOLR-9116] Race condition causing occasional SolrIndexSearcher leak when SolrCore is reloaded

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.5.2, 5.6, 6.0.1, 6.1, 7.0
• Labels: leak searcher
• Assignee: Shalin Shekhar Mangar
• Reporter: Jessica Cheng Mallet
• Created: 16/May/16 16:54
• Updated: 02/Oct/19 17:24
• Resolved: 17/Jun/16 21:44

Description

Fix a leak of SolrIndexSearcher when a SolrCore is reloaded. Added a test to expose this leak when run in many iterations (pretty reliable failure with iters=1K), which passes with the fix (ran iters=10K twice).

The fundamental issue is that when an invocation of SolrCore#openNewSearcher is racing with SolrCore#close, if this synchronized block (https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/core/SolrCore.java#L1611) in openNewSearcher doesn't check for whether or not the core is closed, it can possibly run after the core runs closeSearcher and assign the newly constructed searcher to realtimeSearcher again, which will never be cleaned up. The fix is to check if the SolrCore is closed inside the synchronized block, and if so, clean up the newly constructed searcher and throw an Exception.

Issue Links

• is related to: SOLR-7115 UpdateLog can miss closing transaction log objects.