Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-2462

Using spellcheck.collate can result in extremely high memory usage

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.1
    • Fix Version/s: 3.3, 4.0-ALPHA
    • Component/s: spellchecker
    • Labels:
      None

      Description

      When using "spellcheck.collate", class SpellPossibilityIterator creates a ranked list of every possible correction combination. But if returning several corrections per term, and if several words are misspelled, the existing algorithm uses a huge amount of memory.

      This bug was introduced with SOLR-2010. However, it is triggered anytime "spellcheck.collate" is used. It is not necessary to use any features that were added with SOLR-2010.

      We were in Production with Solr for 1 1/2 days and this bug started taking our Solr servers down with "infinite" GC loops. It was pretty easy for this to happen as occasionally a user will accidently paste the URL into the Search box on our app. This URL results in a search with ~12 misspelled words. We have "spellcheck.count" set to 15.

        Attachments

        1. SOLR-2462.patch
          4 kB
          James Dyer
        2. SOLR-2462_3_1.patch
          5 kB
          Shawn Heisey
        3. SOLR-2462.patch
          6 kB
          James Dyer
        4. SOLR-2462.patch
          6 kB
          James Dyer
        5. SOLR-2462.patch
          8 kB
          James Dyer
        6. SOLR-2462.patch
          8 kB
          James Dyer
        7. SOLR-2462.patch
          10 kB
          James Dyer
        8. SOLR-2462.patch
          11 kB
          James Dyer
        9. SOLR-2462.patch
          14 kB
          James Dyer
        10. SOLR-2462.patch
          15 kB
          James Dyer

          Issue Links

            Activity

              People

              • Assignee:
                rcmuir Robert Muir
                Reporter:
                jdyer James Dyer
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: