Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-3627

Memory leak when using spellchecker with long function query (fq)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Cannot Reproduce
    • Affects Version/s: 3.1, 3.6, 4.0-ALPHA
    • Fix Version/s: None
    • Component/s: search, spellchecker
    • Environment:

      Windows 7 x64
      Tomcat 6.0.26
      Solr 3.1
      Java options: -XX:PermSize=128m -XX:MaxPermSize=128m -Xms1024m -Xmx1024m

      Description

      If spellcheck=true and request to Solr contains too long 'fq', memory is not cleaned, i.e. garbage collector can't clean memory. Fill rate of memory depends on the length of the fq.
      Example request:

      rows=10
      version=2.2
      spellcheck=true
      q=+((shortDesc_en:(hghg)))
      fq=(+contractIds:("contractId1" "contractId2" ... "contractId20000") AND +catalogIds:("catalogId1" "catalogId2" ... "catalogId20000"))
      

      solrconfig.xml:

      ...
      <requestHandler name="standard" class="solr.SearchHandler" default="true">
      	<!-- default values for query parameters -->
      	<lst name="defaults">
      		<str name="echoParams">explicit</str>
      
      		<str name="spellcheck.onlyMorePopular">true</str>
      		
      		<str name="spellcheck.extendedResults">false</str>
      		<str name="spellcheck.collate">true</str>
      		<str name="spellcheck.collateExtendedResults">true</str>
      		
      		<str name="spellcheck.maxCollations">3</str>
      		
      		<str name="spellcheck.maxCollationTries">6</str>
      		
      		<str name="spellcheck.count">3</str>
      	</lst>
      	<arr name="last-components">
      	        <str>spellcheck</str>
      	</arr>
      </requestHandler>
      ...
      
      If spellcheck.collate=false, issue is not reproduced.

      Preliminary research gave following result:
      When contractIds=~20000 and catalogIds=~20000, fill rate of memory is about 60 request.
      When contractIds=~10000 and catalogIds=~10000, fill rate of memory is about 90 request.

        Attachments

        1. SOLR-3627.patch
          0.8 kB
          Sergei Tkachuk
        2. jprofiler_analysis.zip
          8 kB
          Sergei Tkachuk
        3. test-query.txt
          606 kB
          Eugene Viktorovich
        4. solr-perfomance-test(2.5.1).jmx
          1009 kB
          Eugene Viktorovich
        5. solrconfig.xml
          64 kB
          Eugene Viktorovich
        6. javaVisualVM-screen.png
          145 kB
          Eugene Viktorovich
        7. jmeter-screen.png
          94 kB
          Eugene Viktorovich
        8. params.txt
          606 kB
          Eugene Viktorovich

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                volfgang Eugene Viktorovich
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: