[LUCENE-8360] NGramTokenFilter doesn't set posIncAttribute correctly in end()

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 7.4
• Lucene Fields: New
• Assignee: Alan Woodward
• Reporter: Alan Woodward
• Created: 17/Jun/18 20:08
• Updated: 28/Aug/22 15:31
• Resolved: 18/Jun/18 09:22

Description

Found by randomly adding ConditionalTokenFilter to random chains, which calls end() on its delegate when it reaches a token that shouldn't have the delegate applied to it, so that things like position increments and position length are preserved.

This is also the case for EdgeNGramTokenFilter