[LUCENE-3375] processing a synonym in a token stream will remove the following token from the stream

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4, 4.0-ALPHA
• Fix Version/s: 3.4, 4.0-ALPHA
• Component/s: modules/analysis

Environment

solr 4.0 built from trunk today

Solr Specification Version: 4.0.0.2011.08.11.16.00.45
Solr Implementation Version: 4.0-SNAPSHOT 1156711M - simon - 2011-08-11 16:00:45
Lucene Specification Version: 4.0-SNAPSHOT
Lucene Implementation Version: 4.0-SNAPSHOT 1156556 - simon - 2011-08-11 09:33:46
Current Time: Thu Aug 11 21:27:11 EDT 2011
Server Start Time:Thu Aug 11 20:59:11 EDT 2011

• Assignee: Unassigned
• Reporter: Simon Rosenthal
• Created: 12/Aug/11 01:32
• Updated: 28/Aug/22 12:55
• Resolved: 16/Aug/11 16:13

Description

If you do a phrase search on a field derived from a fieldtype with the synonym filter which includes a synonym, the term following the synonym vanishes after synonym expansion.

e.g. http://host:port/solr/corename/select/?q=desc:%22xyzzy%20%20bbb%20pot%20of%20gold%22&version=2.2&start=0&rows=10&indent=on&debugQuery=true (bbb is in the default synonyms file, desc is a "text" fieldtype)

outputs
....
<str name="rawquerystring">desc:"xyzzy bbb pot of gold"</str>
<str name="querystring">desc:"xyzzy bbb pot of gold"</str>
<str name="parsedquery">PhraseQuery(desc:"xyzzy bbbb 1 bbbb 2 of gold")</str>
<str name="parsedquery_toString">desc:"xyzzy bbbb 1 bbbb 2 of gold"</str>
....

You can also see this behavior using the admin console analysis.jsp

Solr 3.3 behaves properly.