[LUCENE-1253] LengthFilter and other TokenFilters that skip tokens ignore relative positionIncrement

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.3.1
• Fix Version/s: 3.1, 4.0-ALPHA
• Component/s: modules/analysis
• Lucene Fields: New, Patch Available
• Assignee: Uwe Schindler
• Reporter: Walter Ferrara
• Created: 30/Mar/08 18:56
• Updated: 28/Aug/22 11:48
• Resolved: 30/Jan/11 18:30

Description

See for reference:
http://www.nabble.com/WordDelimiterFilter%2BLenghtFilter-results-in-termPosition%3D%3D-1-td16306788.html
and http://www.nabble.com/Lucene---Java-f24284.html

It seems that LengthFilter (at least) could produce a stream in which the first Token has a positionIncrement of 0, which make CheckIndex and Luke function "Reconstruct&Edit" to generate exception.

Should something be done to avoid this situation, or could the error be ignored (by allowing Term with a position of -1, and relaxing CheckIndex checks?)

Issue Links

• relates to: LUCENE-1255 CheckIndex should allow term position = -1