Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-1532

File based spellcheck with doc frequencies supplied

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Minor
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: modules/spellchecker
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The file-based spellchecker treats all words in the dictionary as equally valid, so it can suggest a very obscure word rather than a more common word which is equally close to the misspelled word that was entered. It would be very useful to have the option of supplying an integer with each word which indicates its commonness. I.e. the integer could be the document frequency in some index or set of indexes.

      I've implemented a modification to the spellcheck API to support this by defining a DocFrequencyInfo interface for obtaining the doc frequency of a word, and a class which implements the interface by looking up the frequency in an index. So Lucene users can provide alternative implementations of DocFrequencyInfo. I could submit this as a patch if there is interest. Alternatively, it might be better to just extend the spellcheck API to have a way to supply the frequencies when you create a PlainTextDictionary, but that would mean storing the frequencies somewhere when building the spellcheck index, and I'm not sure how best to do that.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              dbowen David Bowen
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: