Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
-
New
Description
FilteringTokenFilter does not handle the position length graph attribute when removing a token from the stream. This doesn't work well with graph token stream that sets position length since removing a token from the stream can invalidate the position length set on the previous tokens.
This issue was first discussed in https://issues.apache.org/jira/browse/LUCENE-4065 but it has a different purpose which is why I am opening a new issue here.