[SOLR-3627] Memory leak when using spellchecker with long function query (fq)

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Cannot Reproduce
• Affects Version/s: 3.1, 3.6, 4.0-ALPHA
• Component/s: search, spellchecker
• Labels: memory_leak performance

Environment

Windows 7 x64
Tomcat 6.0.26
Solr 3.1
Java options: -XX:PermSize=128m -XX:MaxPermSize=128m -Xms1024m -Xmx1024m

• Assignee: Unassigned
• Reporter: Eugene Viktorovich
• Created: 16/Jul/12 13:26
• Updated: 15/Apr/19 11:39
• Resolved: 15/Apr/19 11:39

Description

If spellcheck=true and request to Solr contains too long 'fq', memory is not cleaned, i.e. garbage collector can't clean memory. Fill rate of memory depends on the length of the fq.
Example request:

rows=10
version=2.2
spellcheck=true
q=+((shortDesc_en:(hghg)))
fq=(+contractIds:("contractId1" "contractId2" ... "contractId20000") AND +catalogIds:("catalogId1" "catalogId2" ... "catalogId20000"))

solrconfig.xml:

...
<requestHandler name="standard" class="solr.SearchHandler" default="true">
	<!-- default values for query parameters -->
	<lst name="defaults">
		<str name="echoParams">explicit</str>

		<str name="spellcheck.onlyMorePopular">true</str>
		
		<str name="spellcheck.extendedResults">false</str>
		<str name="spellcheck.collate">true</str>
		<str name="spellcheck.collateExtendedResults">true</str>
		
		<str name="spellcheck.maxCollations">3</str>
		
		<str name="spellcheck.maxCollationTries">6</str>
		
		<str name="spellcheck.count">3</str>
	</lst>
	<arr name="last-components">
	        <str>spellcheck</str>
	</arr>
</requestHandler>
...

If spellcheck.collate=false, issue is not reproduced.

Preliminary research gave following result:
When contractIds=~20000 and catalogIds=~20000, fill rate of memory is about 60 request.
When contractIds=~10000 and catalogIds=~10000, fill rate of memory is about 90 request.

Issue Links

• duplicates: SOLR-2462 Using spellcheck.collate can result in extremely high memory usage