To enable ComplexPhraseQueryParser in Solr 1.4.0

1-) extract ComplexPhrase.zip and run 'mvn package'
2-) copy the ComplexPhrase/target/ComplexPhrase-1.0.jar to solrhome/lib directory
3-) register queryparser to solrhome/conf/solrconfig.xml by adding
<queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" />
4-) enable it by appending &defType=complexphrase to search url.
5-) More permanent usage can be configured in solrconfig.xml
<requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
<lst name="defaults">
<str name="defType">complexphrase</str>
</lst>
</requestHandler>

There is a fixed but forgotten bug in ComplexPhraseQuery at lucene-misc-2.9.1.jar.
This patch includes addition of a jar to trunk/lib directory.
This jar has the corrected version of ComplexPhraseQueryParser.
I couldn't include the corrected version of ComplexPhraseQueryParser in patch because it accesses a package-private field in Lucene's QueryParser.

solr-core maven artifact already contains AbstractSolrTestCase, so it is deleted from test source directory.

To enable ComplexPhraseQueryParser in Solr 1.4.0 - Revisited

Due to the reasons revealed here [1] this plugin should be loaded using the old way [2]

[1] http://search-lucene.com/m/E49gN1naPyh
[2] http://wiki.apache.org/solr/SolrPlugins#The_Old_Way

1-) extract ComplexPhrase.zip and run 'mvn package'
2-) unzip apache-solr-1.4.0.zip and copy ComplexPhrase/target/ComplexPhrase-1.0.jar to apache-solr-1.4.0/lib directory.
3-) create a new apache-solr-1.4.0\dist\apache-solr-1.4.1-dev.war (by running 'ant dist') and use it.

4-) register queryparser to solrhome/conf/solrconfig.xml by adding
<queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" />
5-) enable it by appending &defType=complexphrase to search url.
6-) Alternatively you can add

"s* b*"
7-) More permanent usage can be configured in solrconfig.xml
<requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
<lst name="defaults">
<str name="defType">complexphrase</str>
</lst>
</requestHandler>

Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

A unique token for finding these 240 issues in the future: hossversioncleanup20100527

There is a need for un-ordered proximity search. http://search-lucene.com/m/3W9fj2yzNy82/
Configurable inOrder parameter is added where default behavior is true .

The configuration below can be used to obtain same behavior of PhraseQuery in which order of terms is not important.

 
<queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin">
    <bool name="inOrder">false</bool>
</queryParser>

Hyphen inside the phrase causes exception. e.g. "sulfur-reducing bacteria"
Terje Eggestad's fix is added.

I integrated the patch and its working fine however, there were couple of issues.

One is related to un-ordered proximity which seems to be fixed with the inOrder parameter but its not working for me (doesn't give any error but its still ordered). I will try to get the patch again coz I also merged it in early Nov so maybe it was applied after that.

The other issue is that although proximity search works with phrases BUT its not very accurate e.g. If I want to search "a b" within 10 words of "c" the query would end up being "a b c"~10 but this will also return cases where "a" is not necessarily together with "b". Any scenario where these 3 words are within 10 words of each other will match.

Is it possible in SOLR to do what I mentioned above? Any other patch? Something like " "a b" c "~10...

Note: I was going through Lucene-1486 and there Ahmet mentioned that "Specifically : "(john johathon) smith"~10 " works perfectly. For me it seems there is no difference if I put the parenthesis or not.

Thanks!

Salman, what you are after is nested proximity search which is not currently available in Solr. For alternatives see http://search-lucene.com/m/94ONm1KRuAv1/

Regarding parenthesis inside quotes and inOrder, they are covered in TestCases.

What version of solr did you use?

I am using SOLR 1.4.1 but integrated this patch in early Nov so maybe you committed the inOrder parameter after that?

When you say "Regarding parenthesis inside quotes..." if this works and groups the words in phrase together won't it work for my case e.g. "(a b) c"~10?

I guess if SurroundQuery doesn't use any analyzer it would be very difficult to make the existing queries work (I am using Standard Analyzer).

Use the most recent file which is non-gray color. Also there is a "date attached" info for files.
It works for "(a b) c"~10. This is equivalent to "a c"~10 OR "b c"~10.

SurroundQueryParser does not use Analyzer. It recommended to heavily use wildcard operator instead.
e.g. instead of searching foo bar, you search foo* bar*

But if you are using Standard Analyzer which does not have stemming in it, I think you can use Surround.
You can pre-lowercase your queries etc. You can even pre-analyze your queries since your analyzer does not inject new tokens.
But your queries must be well formed, there is not default operator in this.

But I think it is better to discuss these things in solr/lucene-user mailing list.

I am trying to use CommonGrams with this patch but doesn't seem to work.

If I don't add

it simply does it the old way i.e. ignore CommonGrams.

Can you please help how can I combine both these features?

I tried the patch with latest non-grayed file but still inOrder doesn't seem to have any impact.

Results for "a b"~5 and "b a"~5 are still different.

Also any feedback about CommonGrams integration?

Thanks a lot for all the help!

When you add &debugQuery=on to your search URL, you should see something like in the debug section:
spanNear([text:a, text:b], 5, false) , false here means un-ordered phrase query. Do you see it?

I will look into CommonGrams this weekend.

Hi

I am having same issue (inOrder does not seem to work) with

, in the debug section it shows spanNear([text:a, text:b], 5, true),
my solrconfig.xml looks like

<queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" >
<bool name="inOrder">false</bool>
</queryParser>

Hi Ahmet

I have read on mailing lists dated "Mon, 20 Jul 2009" that you had merged the surround query parser with solr.
I have tried that by downloading jar file for surround query parser. and then pasting that jar file in web-inf/lib
and configured query parser in solrconfig.xml as
<queryParser name="SurroundQParser" class="org.apache.lucene.queryParser.surround.parser.QueryParser"/>

then web page the following exception comes

org.apache.solr.common.SolrException: Error Instantiating QParserPlugin, org.apache.lucene.queryParser.surround.parser.QueryParser is not a org.apache.solr.search.QParserPlugin

can u guide what i m doing wrong....

Regards
Ahsan

Ahmet,

I would be waiting for your response on CommonGrams. Would be grateful if you can look into it this weekend. Thanks!

Although I would be asking this question on the mailing list as well but since its related to this patch so I wanted to check if this patch would work fine with SurroundQueryParser or if Surround does it itself? This patch functionality is really important for me.

Thanks a lot!

Any updates on integration with CommonGrams? Thanks

Reminder! Any updates regarding integration with CommonGrams? Thanks

Bulk move 3.2 -> 3.3

Will the complexphrase search works fine with e.g. GOK:"PXB 80?", it will throw an exception if there is no space present, e.g. GOK:"PXB80?". The exception is:
Unknown query type "org.apache.lucene.search.WildcardQuery" found in phrase query string "PXB80?"

Update for solr 3.3.0

• Download apache-solr-3.3.0-src.tgz
• Download most latest ComplexPhrase.zip
• 'mvn package' will generate 3 files under target folder
  copy them to apache-solr-3.3.0/solr/lib/
  • cp target/ComplexPhrase-* Downloads/apache-solr-3.3.0/solr/lib/
• call 'ant clean dist' to create a new apache-solr-3.3-SNAPSHOT.war file under solr/dist folder

3.4 -> 3.5

I have installed complexphrase parser and followed all instructions for installation... when i run with defType=complexphrase , i get below error.
HTTP Status 500 - luceneMatchVersion java.lang.NoSuchFieldError: luceneMatchVersion at org.apache.solr.search.SolrComplexPhraseQueryParser

how to solve this. i am new to solr

The class org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser was committed and it's available in "modules/queryparser". How should this patch proceed in order to make that QParser available in Solr? Should it be added to the trunk? as a contrib?

Committed o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser does not work with non-default fields. Several Lucene users raised this issue on mailing lists. Mark Harwood said the following on LUCENE-1486 which is still unresolved. However it didn't get any attention.

Fixing this would require changing the package name of ComplexPhraseQueryParser or changing the visibility of "field" in the QueryParser base class to "protected".
Anyone have any strong feelings about which of these is the most acceptable?

That's why attachment of this issue is not consuming committed o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser and released as a solr-plugin.

That's true, but probably this Jira should focus in using the available ComplexPhraseQueryParser and there should be another Jira (a Lucene one) to fix that problem. The parser was moved for a reason and if we use a duplicated version of the parser in a different package (the old one) this will never be committed.

I imagined that LUCENE-1486 will be closed/fixed in the future, hopefully including the non default field patch. Are you saying that non default field problem should be handled in a separate issue (other than LUCENE-1486) ?

It might be LUCENE-1486 or a new Lucene Jira, but what I say is that it should not be fixed in this issue. What I'm proposing is to only provide the necessary classes in Solr to be able to work with the current version of ComplexPhraseQueryParser, even if it is limited in some aspects. Once the ComplexPhraseQueryParser is improved we'll get those improvements in Solr (either automatically or by improving something from this side).

path for trunk

to make inOrder parameter configurable

to enable non-default field searches

In my opinion, we should not be relying on un-committed Lucene code, as it may change or even never get committed. I think this Jira should only consider the current ComplexPhraseQueryParser and we can create a new Solr Jira that depends on LUCENE-3758 for the order/unordered query.

I'm attaching an alternative patch with what I propose:
It works fine with what's committed today in Lucene, it doesn't require LUCENE-3758 committed to compile or LUCENE-1486 committed to run the tests, however, it
has the limitations of the currently committed ComplexPhraseQueryParser.
I have some commented testUnorderedPhraseQuery. It is commented because it currently doesn't run to the the limitations of LUCENE-3758
testFieldedQuery is also commented, as it doesn't run until LUCENE-1486 is committed.
I added testAutogeneratingPhraseQueries, but it is also commented as hyphenated queries don't work when "autogenerate phrase queries" is set to true. This is something we should continue working on in this issue.
I would like to hear opinions on how to continue working.

Did anybody got the chance to see these two patches? are there any more opinions on this?

Bulk changing fixVersion 3.6 to 4.0 for any open issues that are unassigned and have not been updated since March 19.

Email spam suppressed for this bulk edit; search for hoss20120323nofix36 to identify all issues edited

I am upgrading Solr to version 3.6 from version 1.4.1 and applied this patch. The Solr is built successfully. After indexing some document I performed a wildcard phrase query "weekly repo*", no documents were returned. However, by appending "&defType=complexphrase" the solr returned the document containing "weekly report". It seems that the complex phrase is not taken as default search handler.
Also, when restarting solr I discovered following error in log file as:

SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.search.ComplexPhraseQParserPlugin'
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:394)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:419)
at org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:441)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1612)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1606)
at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1639)
at org.apache.solr.core.SolrCore.initQParsers(SolrCore.java:1556)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:555)
at org.apache.solr.core.CoreContainer.create(CoreContainer.java:483)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335)
at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219)
at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161)
at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
at org.mortbay.jetty.Server.doStart(Server.java:224)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.mortbay.start.Main.invokeMain(Main.java:194)
at org.mortbay.start.Main.start(Main.java:534)
at org.mortbay.start.Main.start(Main.java:441)
at org.mortbay.start.Main.main(Main.java:119)
Caused by: java.lang.ClassNotFoundException: org.apache.solr.search.ComplexPhraseQParserPlugin
at java.net.URLClassLoader$1.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.net.FactoryURLClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:378)
... 37 more

I would like some help regarding the use of this patch in new solr.

Hi everyone,

Sorry, but I'm green to this stuff. How do I "apply/install" a "*.patch" file?

I downloaded and successfully built (well, "packaged", via mvn) the ComplexPhrase.zip from Jul-2011.

I then downloaded the 'SOLR-1604-alternative.patch' from Feb-2012. I can open and view it via a text editor...but I have no idea what to do to "apply" it? I'm working on a RH Linux box at the moment.

Can anyone guide me and/or point me in the right direction please?

Thanks!
Mike

There are two separate ways to enable this functionality.

• You can consume *.zip attachments as a solr plugin. Which does not require source code modification, but this particular case requires re-creating solr.war. http://wiki.apache.org/solr/SolrPlugins

• *.patch files contains source code modifications. http://wiki.apache.org/solr/HowToContribute#Working_With_Patches

bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment

rmuir20120906-bulk-40-change

Hello all,

Could I get the grammar file (.jj file) for ComplexPhrase one - It's not there as part of the patch/zip file.

Thanks,
Phani.

Could I get the grammar file (.jj file) for ComplexPhrase one - It's not there as part of the patch/zip file.

It does not have a separate grammar file. It just extends QueryPaser.

removing 4.0 from fix version since this depends on LUCENE-1486 which is not already in the 4x branch .. can revist if there is movement on LUCENE-1486

I have noticed that LUCENE-1486 will be fixed in v4.1, and I have found that this patch can not work with solr 4.0. Is there new patch for 4.0?

If there would be a working patch, it could be in 4.0. It was just moved out because nobody took care. It's your turn!

this supposed to work with solr 4.0

Hi, Arslan, when using your patch for 4.0, I got a Exception: java.lang.IllegalArgumentException: Unknown query type "org.apache.lucene.search.PrefixQuery" found in phrase query string "mp*", my query is q=subject:"mp*"&wt=json. This exception was thrown at QueryComponent#process.

Jun Wang query string "mp*" is somehow invalid. Use &q=mp* instead. This parser expects at least two clauses inside quotes.

Includes README.txt that contain instruction for Solr 4.0.0

Hi, I downloaded last version of ComplexPhrase (24/Oct/12 02:30) but have problem with junit test. Here is error log :

test(org.apache.solr.search.ComplexPhraseQParserPluginTest)  Time elapsed: 0.191 sec  <<< ERROR!
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([4BF35CC9C13F3B15:C3A763136FC356ED]:0)
	at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:283)
	at org.apache.solr.search.ComplexPhraseQParserPluginTest.test(ComplexPhraseQParserPluginTest.java:158)
	// nothing interest here
Caused by: java.lang.IllegalArgumentException: Unknown query type "org.apache.lucene.search.ConstantScoreQuery" found in phrase query string "jo* [sma TO smz]"
	at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser$ComplexPhraseQuery.rewrite(ComplexPhraseQueryParser.java:297)
	at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:599)
	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:646)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:280)
	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1385)
	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1260)
	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:390)
	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:411)
	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:206)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:364)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:346)
	at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:273)
	... 41 more

Is it error on my side (I didn't change anything)? Or some crucial error?

I also see the error Roman posted, with the latest version

This is ComplexPhrase project based on the version submitted on 21/Jul/11. It compiles and runs under solr 3.4. I have uncommented the tests in /org/apache/solr/search/ComplexPhraseQParserPluginTest.java and they passed.

Hello! Great work!

I have two questions:

1) What would it take to incorporate phrase searches into this extended query parser?
"\"a b\" c"~100
that is, "a b" (phrase search) is found in that order and exactly side by side <=100 tokens away from c.

2) does this implementation support the Boolean operators, like AND, OR, NOT (at least OR and NOT are supported as far as I can see)? Can they be nested?

Hi Ahmet and fellow ComplexPhrase users,

We have been using the ComplexPhraseQueryParser for a number of years with our SOLR installation, but we were hoping to upgrade to SOLR 4.1 and I was just wondering if ComplexPhrase works with the latest SOLR release?

We are still on SOLR 1.4.1 so keen to take advantage of all that SOLR 4.1 can offer, but we can't go there without ComplexPhrase

Much thanks if anyone has anyone has any thoughts.

Bring to 4.2.1 version

Hi Ahmet
I have installed complexphrase parser and followed all instructions for installation...
Everything is fine except getting NULL Pointer exception when I start JBOSS.

19:46:03,909 ERROR [SolrCore] java.lang.NullPointerException
at org.apache.solr.schema.IndexSchema$DynamicReplacement$DynamicPattern$NameEndsWith.matches(IndexSchema.java:960)
at org.apache.solr.schema.IndexSchema$DynamicReplacement.matches(IndexSchema.java:974)
at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1221)
at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:418)
at org.apache.lucene.analysis.AnalyzerWrapper.initReader(AnalyzerWrapper.java:81)
at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175)
at org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:507)
at org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:495)
at org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1109)
at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:121)
at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.parse(ComplexPhraseQueryParser.java:107)
at org.apache.solr.search.ComplexPhraseQParser.parse(ComplexPhraseQParserPlugin.java:108)
at org.apache.solr.search.QParser.getQuery(QParser.java:142)
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:142)
at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64)
at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1693)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)

Thanks in advance.

Hi ettey siva, what version of solr did you use?

Hi Ahmet
We are using solr 4.4.
Issue was resolved by uncommenting <defaultSearchField>text</defaultSearchField> in schema.xml
Thank you

I have a question about upgrading solr with this plugin. Because the plugin has to be built into solr.war, I need to go through the process each time I upgrade solr. It's rather cumbersome. Does anyone have a better solution?

Rebecca Tang you can define a solr core (even for a single index) and use its lib directory to copy the complex phrase parser jar.

https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml

HTH

Rebecca, for the time being thats the only way to use it. Lib directory fails as explained here : http://search-lucene.com/m/E49gN1naPyh
The problem is fielded queries do not work with current committed CPQP. If LUCENE-1486 gets fixed then this plugin can consume original CPQP.

Ahmet,
I am using solr 4.3.1. do i still need to apply this patch ? if yes please tell me what patch to apply, I see lot of patches here, not sure which patch for what version of solr.

Hi everyone,

Does this plugin supports date searches?
I noticed that doing something like this:
q=Project.CommenceDate:[2006-08-01T00:00:00Z%20TO%202006-08-01T00:00:00Z]

gives no results (When I use edismax it does return some results).
I also tried:
q=Project.CommenceDate:2006-08-01T00:00:00Z

but I get this error:
Invalid Date String:'2006-08-01T00'

Thanks,
Randall Rosales

Two things:

1> dots in field names are risky, in general you should only use valid Java variable naming conventions.
2> How are you submitting this? Because ':' is the delimiter between fields and values, you need to escape this line if you're submitting this via a browser, as 00\:00\:00Z

BTW, you'll get faster responses to this kind of question if you raise it on the user's list, more eyes see it.

Hey,
I have build Complex Phrase 4.2.1 and followed the steps given in first comment. I am using this parser with solr 4.6 version and when I executed URL by appending &defType=complexphrase I got an error given below. Any help would be appreciated thanks.
java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/solr/search/SolrIndexReader at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:724) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:434) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.NoClassDefFoundError: org/apache/solr/search/SolrIndexReader at org.apache.solr.search.ComplexPhraseQParserPlugin.createParser(ComplexPhraseQParserPlugin.java:56) at org.apache.solr.search.QParser.getParser(QParser.java:347) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:141) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) ... 16 more Caused by: java.lang.ClassNotFoundException: org.apache.solr.search.SolrIndexReader at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.net.FactoryURLClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 24 more

Hi Anonymous, It is possible that some api has changed. Tomorrow I will attach solr4.6 compatible one. In mean time you could give LUCENE-5205 a try. It has all the functionalities along with additional one. And its author Tim Allison responds very fast to any comments/questions/improvements/bugs etc. By the way SOLR-5410 is the wrapper for LUCENE-5205

followed the steps given in first comment.

Anonymous, There is a ReadMe.txt in zip file. It has correct instructions.

In ReadMe.txt they are putting jar file in to WEB_INF/lib dir and zipping it to new solr.war which I guess pretty much the same.

It is mandatory because there is a package private member access and class loader problem.
"A class loaded by one class loader can't access package-visible members in a class loaded by a different
classloader even if they are nominally in the same package." [1]

[1]http://www.pubbs.net/grails/200911/37981/

Now getting this error
java.lang.NullPointerException at java.io.StringReader.<init>(Unknown Source) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:117) at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.parse(ComplexPhraseQueryParser.java:107) at org.apache.solr.search.ComplexPhraseQParser.parse(ComplexPhraseQParserPlugin.java:108) at org.apache.solr.search.QParser.getQuery(QParser.java:142) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:142) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

Issue Resolved Thanks for helping

Solr 4.7.0 compatible version.

I need to look at this, so I'm assigning it to myself. That said, my knowledge of this is sparse, so I'll have a bit of a learning curve. Anyone else who wants to tackle this please feel free to take it.

Ahmet:
Thnkas! This couldn't have come at a better time, I was dreading dealing with this.

Hi Erick Erickson thanks into looking into this. Let me summarise what have been so far.

1) Consume existing ComplexPhraseQueryParser from lucene code base. Similar to lucene.SurroundQueryParser and solr.SurroundQParserPlugin relationship. This is the easiest one from solr perspective, in a way that it does not require any lucene code change. However fielded queries won't work in this solution. And you cannot set inOrder=

parameter for proximity.

2) First fix LUCENE-1486 fielded query problem. Patch already available at LUCENE-1486. Forget about inOrder parameter.

3) First fix both LUCENE-1486 and LUCENE-3758.

Please let me know which is the preferred one.

Probably it make sense to base plug-in on solr sources.
Attached patch contains: org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser and org.apache.solr.search.ComplexPhraseQParserPlugin in solr and lucene sources respectively.
Made some code cleanup.
Added tests on lucene side for org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.
Renamed test resources for solr plugin test.
I continue working on name:jo* prefix query issue when Highlighter return empty string instead of highlight matched term (TODO added).

Probably it make sense to base plug-in on solr sources.

Can you please explain what do you mean by that?

I mean 2 things:
1. Put plug-in sources in Solr codebase (not like separate lib jar)
2. Make org.apache.solr.search.ComplexPhraseQParserPlugin as built-in standard plug-ins

Added patch fixing name:jo* issue with highlighter.
Tests in org.apache.solr.search.ComplexPhraseQParserPluginTest uncommented and passing.

OK, I was looking around the patch and think I understand at least some of what's going on. To drive this forward, I need a couple of things:

1> Vitaliy and Ahmet to resolve the two patches and let me know what the right one to use is. BTW, Vitaliy, please use "svn diff" or the equivalent Git command to create patches. Zipped up sources are much harder to work with.

2> Some idea of a roadmap from here. Straw-man proposal:
2a> Close 1486 and open a new JIRA if there's a fix for that if necessary. It looks to me like this patch can be committed without 1486 and we'll generate a separate fix.
2b> commit 3758, and remove inOrder from this patch, then commit this patch.
2c> I've assigned these to myself so I don't lose track of them. I'll look desperately for cycles to work on them . But I have a couple of long plane flights in my future...

3> Of course we need to document the syntax and behavior here, Cassandra Targett can probably point us in the right direction for doing this right by putting it in the new documentation!

4> I'm also curious what we know now in terms of performance, resource requirements, that kind of stuff.

5> I notice there's a patch labeled as having to do with license stuff. What's up there? Is this just putting the headers in the source files?

5> Anything else? Does anyone out there object to moving forward with this?

Here are some clarification regarding zipped attachments :

Zipped attachments are not meant for source code inclusion but for to be consumed as solr plugin. They will never be committed. Mainly because zipped version(s) include a duplicate code from lucene code base. Duplicated class is org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser. Duplication is done for two reasons :

• To enable fielded queries. this duplicate code changes package name to org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser. Originally Somehow this feature forgotten accidentally in LUCENE1468, while committing lucene.ComplexPhraseQueryParser. After that commit, package name changed from classic to complexPhrase. For this fix it needs to access a field from super class. After realizing this, chancing this fields visibility to protected is accepted by lazy consensus. This is the patch for this.
• To enable ability change inOrder parameter. In original lucene code inOrder parameter is barcoded to true inSpanNearQuery classes. Separate jira for this is LUCENE-3758.

By the way, why LUCENE-1486 is re-opened is a mystery. It is not re-opened because of unforgotten non-default patch.

Vitaliy and Ahmet to resolve the two patches and let me know what the right one to use is.

none of them actually. They include source code (ComplexPhraseQueryParser.java) duplication from lucene. I will attach a patch that consumes lucene's ComplexPhraseQueryParser created against trunk.

Close 1486 and open a new JIRA if there's a fix for that if necessary. It looks to me like this patch can be committed without 1486 and we'll generate a separate fix.

+1. Yes this patch can be committed without LUCENE-1486. +1 for closing LUCENE-1486 given that it is re-opened mysteriously. +1 for creating a separate jira for this functionality just because it is less confusing.

commit 3758, and remove inOrder from this patch, then commit this patch.

Request ability change inOrder parameter came from a user. Robert had this comment about this.

I notice there's a patch labeled as having to do with license stuff.

This attachment is old. I accidentally forget selection 'ASF inclusion" radio box then. Jira weren't displaying feather icon for that. After that incident jira had removed that radio button selection option. Attachments are ASF granted by default now. That file is renamed automatically by infra.

This is solr-only patch (solr/core/src/) and does not touch lucene code case. It adds two new java classes (ComplexPhraseQParserPlugin and TestComplexPhraseQParserPlugin) and consumes o.a.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser

Added entry to CHANGES.txt for Solr

QueryEqualityTest added.

following commands pass for me with last patch.

• ant -Dtests.disableHdfs=true -Dtests.badapples=false test
• ant -Dtestcase=QueryEqualityTest test
• ant -Dtestcase=TestComplexPhraseQuery test
• ant -Dtestcase=TestComplexPhraseQParserPlugin test

Commit 1578200 from Erick Erickson in branch 'dev/trunk'
[ https://svn.apache.org/r1578200 ]

SOLR-1604: Wildcards, ORs etc inside Phrase Queries or 'ComplexPhraseQueryParser support in Solr'

Commit 1578218 from Erick Erickson in branch 'dev/branches/branch_4x'
[ https://svn.apache.org/r1578218 ]

SOLR-1604: Wildcards, ORs etc inside Phrase Queries or 'ComplexPhraseQueryParser support in Solr'

OK, quite a bit has been checked in for this, but I think it's all done. Let's see what happens now!

OK, to finish this off, we need a Wiki/Confluence page, calling for volunteers:

Some points that should be mentioned I think:
> how to set up/use (simple really, defType)
> A number of examples
> inOrder="true|false" as a local param mentioned
> Anyone's experience with how it performs, especially with things like single-letter wildcards (e.g. "j* smith")

Thanks Erick Erickson for committing this! Here is what I compiled from README.txt from zip distro.

After indexing example documents under example/exampledocs via 'java -jar post.jar *.xml' utility

The query string

q=manu:"a* c*"&defType=complexphrase

or

q={!complexphrase inOrder=true}manu:"a* c*" 

will return :

http://localhost:8983/solr/collection1/select?q=manu:%22a*%20c*%22&defType=complexphrase&fl=manu

<doc>
  <str name="manu">Apple Computer Inc.</str>
</doc>
<doc>
  <str name="manu">ASUS Computer Inc.</str>
</doc>

inOrder Parameter can be set in two ways.

1) Its default value is true. If you want to set it to false in a permanent way : register query parser with a different name in solrconfig.xml

 <!-- Un-ordered complex phrase query parser -->
  <queryParser name="unorderedcomplexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin">
    <bool name="inOrder">false</bool>
  </queryParser>

2) At query time via local params.

q={!complexphrase inOrder=false df=name}"bla* pla*"

To mix ordered and unordered clauses in the same query.

+_query_:"{!complexphrase inOrder=true}manu:\"a* c*\""  +_query_:"{!complexphrase inOrder=false df=name}\"bla* pla*\""  

Limitations

maxBooleanClauses

You may need to increase

<maxBooleanClauses>1024</maxBooleanClauses>

according to index size in solrconfig.xml because

"a* c*"

is expanded into SpanNearQuery

spanNear([spanOr([manu:a, manu:america, manu:apache, manu:apple, manu:asus, manu:ati]), spanOr([manu:canon, manu:co, manu:computer, manu:corp, manu:corsair])], 0, false)

Stopwords

Lets say we add the, up, to to collection1/conf/stopwords.txt file and re-index example docs.
While

q=features:"Stores up to 15,000"

returns "Stores up to 15,000 songs, 25,000 photos, or 150 hours of video",

q=features:"sto* up to 15*"&defType=complexphrase 

does not return that document because SpanNearQuery has no good way to handle stopwords in a way analagous to PhraseQuery. It is recommended not to use stopword elimination with this query parser.

Hey,
I am using ComplexPhrase 4.2.1 with solr 4.7.0
Is there any difference between ComplexPhrase 4.7.0 and ComplexPhrase 4.2.1?
And also
Can I run this type of query "hello world h*" with this parser?

Hi Anonymous, there is no difference between 4.7.0 and 4.2.1. Committed one has local param addition. You can set inOrder parameter using http://wiki.apache.org/solr/LocalParams

And yes "hello world h*" is perfectly fine.

Thanks Ahmet,
Yeah I saw that using debugQuery On and found that even phrase query is executed using "Spannear query" the addition of wildcard makes it to use "spanOR" inside "spannear query".
One more Question:
when spannear is given multiple words like spannear(hello,world,hello) with slop "0" how it is working inside to find all the documents containing all above terms adjacent to eachother?

when spannear is given multiple words like spannear(hello,world,hello) with slop "0"

With inOrder=true or inOrder=false?

I am sure what happens, when you use the same term twice. I remember someone asked similar question on lucene list. http://search-lucene.com/m/SnDCWotpUB1 Span*Query family less used so there can be gotchas. Can you please tell us this treat is related your use case?

inorder=true
I am asking for general use case there can be different terms also.

Let's move this discussion to the user's list. We're in to "how to use this" discussion I think. That way we'll get a wider audience for this.

Hi Erick Erickson, yes you are right about user's mailing list.
By the way did you have a chance look into documentation candidate I wrote?

Ahmet Arslan About the doc candidate. I took a quick glance at it, and I say go for it.

This looks like a good intro. Any shortcomings will become immediately
apparent and we'll fix it. The only documentation I really object to is false
documentation .

So feel free to put it up on the Wiki. If I could ask you to, after you do create a JIRA
and assign it to me to put the data over on the CWiki...

Thanks!

feel free to put it up on the Wiki.

Done. http://wiki.apache.org/solr/ComplexPhraseQueryParser

create a JIRA and assign it to me to put the data over on the CWiki…

Done. SOLR-5926

Close issue after release of 4.8.0