[LUCENE-2605] queryparser parses on whitespace

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 6.2
• Component/s: core/queryparser
• Assignee: Steven Rowe
• Reporter: Robert Muir
• Created: 17/Aug/10 03:30
• Updated: 22/Nov/24 01:40
• Resolved: 05/Jul/16 22:59

Description

The queryparser parses input on whitespace, and sends each whitespace separated term to its own independent token stream.

This breaks the following at query-time, because they can't see across whitespace boundaries:

• n-gram analysis
• shingles
• synonyms (especially multi-word for whitespace-separated languages)
• languages where a 'word' can contain whitespace (e.g. vietnamese)

Its also rather unexpected, as users think their charfilters/tokenizers/tokenfilters will do the same thing at index and querytime, but
in many cases they can't. Instead, preferably the queryparser would parse around only real 'operators'.

Issue Links

• breaks: LUCENE-7472 MultiFieldQueryParser.getFieldQuery() drops queries that are neither BooleanQuery nor TermQuery
• is related to: LUCENE-7315 Flexible "standard" query parser parses on whitespace
• is related to: SOLR-9185 Solr's edismax and "Lucene"/standard query parsers should optionally not split on whitespace before sending terms to analysis
• is related to: SOLR-4381 Query-time multi-word synonym expansion
• relates to: SOLR-5379 Query-time multi-word synonym expansion