[SOLR-5152] EdgeNGramFilterFactory deletes token

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 4.4
• Assignee: Unassigned
• Reporter: Christoph Lingg
• Created: 14/Aug/13 15:14
• Updated: 02/Aug/18 14:07
• Resolved: 12/Jun/18 19:16

Description

I am using EdgeNGramFilterFactory in my schema.xml

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <!-- ... -->
    <filter class="solr.EdgeNGramFilterFactory" minGramSize="2" maxGramSize="10" side="front" />
  </analyzer>
</fieldType>

Some tokens in my index only consist of one character, let's say R. minGramSize is set to 2 and is bigger than the length of the token. I expected the NGramFilter to left R unchanged but in fact it is deleting the token.

For my use case this interpretation is undesirable, and probably for most use cases too!?

Issue Links

• duplicates: LUCENE-7960 NGram filters -- preserve the original token when it is outside the min/max size range
• is duplicated by: SOLR-5332 Add "preserve original" setting to the EdgeNGramFilterFactory