Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The current synonymsfilter uses a lot of ram and cpu, especially at build time.

      I think yesterday I heard about "huge synonyms files" three times.

      So, I think we should use an FST-based structure, sharing the inputs and outputs.
      And we should be more efficient with the tokenStream api, e.g. using save/restoreState instead of cloneAttributes()

        Attachments

        1. synonyms.zip
          575 kB
          Robert Muir
        2. LUCENE-3233.patch
          16 kB
          Robert Muir
        3. LUCENE-3233.patch
          48 kB
          Michael McCandless
        4. LUCENE-3233.patch
          44 kB
          Robert Muir
        5. LUCENE-3233.patch
          52 kB
          Michael McCandless
        6. LUCENE-3233.patch
          54 kB
          Michael McCandless
        7. LUCENE-3233.patch
          73 kB
          Robert Muir
        8. LUCENE-3233.patch
          78 kB
          Robert Muir
        9. LUCENE-3233.patch
          80 kB
          Robert Muir
        10. LUCENE-3233.patch
          83 kB
          Robert Muir
        11. LUCENE-3233.patch
          89 kB
          Michael McCandless
        12. LUCENE-3233.patch
          95 kB
          Robert Muir
        13. LUCENE-3233.patch
          91 kB
          Michael McCandless
        14. LUCENE-3233.patch
          94 kB
          Michael McCandless
        15. LUCENE-3233.patch
          252 kB
          Robert Muir
        16. LUCENE-3233.patch
          252 kB
          Robert Muir
        17. LUCENE-3233.patch
          248 kB
          Michael McCandless
        18. LUCENE-3223.patch
          29 kB
          Michael McCandless

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rcmuir Robert Muir
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: