Solr
  1. Solr
  2. SOLR-2462

Using spellcheck.collate can result in extremely high memory usage

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 3.1
    • Fix Version/s: 3.3, 4.0-ALPHA
    • Component/s: spellchecker
    • Labels:
      None

      Description

      When using "spellcheck.collate", class SpellPossibilityIterator creates a ranked list of every possible correction combination. But if returning several corrections per term, and if several words are misspelled, the existing algorithm uses a huge amount of memory.

      This bug was introduced with SOLR-2010. However, it is triggered anytime "spellcheck.collate" is used. It is not necessary to use any features that were added with SOLR-2010.

      We were in Production with Solr for 1 1/2 days and this bug started taking our Solr servers down with "infinite" GC loops. It was pretty easy for this to happen as occasionally a user will accidently paste the URL into the Search box on our app. This URL results in a search with ~12 misspelled words. We have "spellcheck.count" set to 15.

      1. SOLR-2462.patch
        15 kB
        James Dyer
      2. SOLR-2462.patch
        14 kB
        James Dyer
      3. SOLR-2462.patch
        11 kB
        James Dyer
      4. SOLR-2462.patch
        10 kB
        James Dyer
      5. SOLR-2462.patch
        8 kB
        James Dyer
      6. SOLR-2462.patch
        8 kB
        James Dyer
      7. SOLR-2462.patch
        6 kB
        James Dyer
      8. SOLR-2462.patch
        6 kB
        James Dyer
      9. SOLR-2462_3_1.patch
        5 kB
        Shawn Heisey
      10. SOLR-2462.patch
        4 kB
        James Dyer

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Robert Muir
              Reporter:
              James Dyer
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development