Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-3233

HuperDuperSynonymsFilterâ„¢

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.4, 4.0-ALPHA
    • None
    • None
    • New

    Description

      The current synonymsfilter uses a lot of ram and cpu, especially at build time.

      I think yesterday I heard about "huge synonyms files" three times.

      So, I think we should use an FST-based structure, sharing the inputs and outputs.
      And we should be more efficient with the tokenStream api, e.g. using save/restoreState instead of cloneAttributes()

      Attachments

        1. LUCENE-3223.patch
          29 kB
          Michael McCandless
        2. LUCENE-3233.patch
          248 kB
          Michael McCandless
        3. LUCENE-3233.patch
          252 kB
          Robert Muir
        4. LUCENE-3233.patch
          252 kB
          Robert Muir
        5. LUCENE-3233.patch
          94 kB
          Michael McCandless
        6. LUCENE-3233.patch
          91 kB
          Michael McCandless
        7. LUCENE-3233.patch
          95 kB
          Robert Muir
        8. LUCENE-3233.patch
          89 kB
          Michael McCandless
        9. LUCENE-3233.patch
          83 kB
          Robert Muir
        10. LUCENE-3233.patch
          80 kB
          Robert Muir
        11. LUCENE-3233.patch
          78 kB
          Robert Muir
        12. LUCENE-3233.patch
          73 kB
          Robert Muir
        13. LUCENE-3233.patch
          54 kB
          Michael McCandless
        14. LUCENE-3233.patch
          52 kB
          Michael McCandless
        15. LUCENE-3233.patch
          44 kB
          Robert Muir
        16. LUCENE-3233.patch
          48 kB
          Michael McCandless
        17. LUCENE-3233.patch
          16 kB
          Robert Muir
        18. synonyms.zip
          575 kB
          Robert Muir

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            rcmuir Robert Muir
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment