[SOLR-2628] use of FST for SynonymsFilterFactory and synonyms.txt

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 3.4, 4.0-ALPHA
• Component/s: Schema and Analysis
• Labels: suggestion

Environment

Linux

• Assignee: Dawid Weiss
• Reporter: Bernd Fehling
• Created: 01/Jul/11 07:38
• Updated: 13/Jan/12 19:35
• Resolved: 01/Jul/11 10:37

Description

Currently the SynonymsFilterFactory builds up a memory based SynonymsMap.
This can generate huge maps because of the permutations for synonyms.

Now where FST (finite state transducer) is introduced to lucene this could also be used for synonyms.
A tool can compile the synoynms.txt file to a binary automaton file which can then be used
with SynoynmsFilterFactory.

Advantage:

• faster start of solr, no need to generate SynonymsMap
• faster lookup
• memory saving

Issue Links

• duplicates: LUCENE-3233 HuperDuperSynonymsFilter™