Solr
  1. Solr
  2. SOLR-1604

Wildcards, ORs etc inside Phrase Queries

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 4.8, 5.0
    • Component/s: query parsers, search
    • Labels:
      None

      Description

      Solr Plugin for ComplexPhraseQueryParser (LUCENE-1486) which supports wildcards, ORs, ranges, fuzzies inside phrase queries.

      1. SOLR-1604-alternative.patch
        31 kB
        Tomás Fernández Löbbe
      2. SOLR-1604.patch
        28 kB
        Ahmet Arslan
      3. SOLR-1604.patch
        26 kB
        Ahmet Arslan
      4. SOLR-1604.patch
        54 kB
        Vitaliy Zhovtyuk
      5. SOLR-1604.patch
        54 kB
        Vitaliy Zhovtyuk
      6. SOLR-1604.patch
        21 kB
        Ahmet Arslan
      7. SOLR-1604.patch
        20 kB
        Erick Erickson
      8. SOLR1604.patch
        22 kB
        Ahmet Arslan
      9. ComplexPhraseQueryParser.java
        17 kB
        Ahmet Arslan
      10. ComplexPhrase-4.7.zip
        24 kB
        Ahmet Arslan
      11. ComplexPhrase-4.2.1.zip
        24 kB
        Ahmet Arslan
      12. ComplexPhrase.zip
        20 kB
        Ahmet Arslan
      13. ComplexPhrase.zip
        16 kB
        Ahmet Arslan
      14. ComplexPhrase.zip
        17 kB
        Ahmet Arslan
      15. ComplexPhrase.zip
        36 kB
        Ahmet Arslan
      16. ComplexPhrase.zip
        25 kB
        Ahmet Arslan
      17. ComplexPhrase.zip
        27 kB
        Ahmet Arslan
      18. ComplexPhrase_solr_3.4.zip
        25 kB
        Dmitry Kan
      19. ASF.LICENSE.NOT.GRANTED--ComplexPhrase.zip
        16 kB
        Ahmet Arslan

        Issue Links

          Activity

          Hide
          Ahmet Arslan added a comment -

          To enable ComplexPhraseQueryParser in Solr 1.4.0

          1-) extract ComplexPhrase.zip and run 'mvn package'
          2-) copy the ComplexPhrase/target/ComplexPhrase-1.0.jar to solrhome/lib directory
          3-) register queryparser to solrhome/conf/solrconfig.xml by adding
          <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" />
          4-) enable it by appending &defType=complexphrase to search url.
          5-) More permanent usage can be configured in solrconfig.xml
          <requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
          <lst name="defaults">
          <str name="defType">complexphrase</str>
          </lst>
          </requestHandler>

          Show
          Ahmet Arslan added a comment - To enable ComplexPhraseQueryParser in Solr 1.4.0 1-) extract ComplexPhrase.zip and run 'mvn package' 2-) copy the ComplexPhrase/target/ComplexPhrase-1.0.jar to solrhome/lib directory 3-) register queryparser to solrhome/conf/solrconfig.xml by adding <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" /> 4-) enable it by appending &defType=complexphrase to search url. 5-) More permanent usage can be configured in solrconfig.xml <requestHandler name="standard" class="solr.StandardRequestHandler" default="true"> <lst name="defaults"> <str name="defType">complexphrase</str> </lst> </requestHandler>
          Hide
          Ahmet Arslan added a comment -

          There is a fixed but forgotten bug in ComplexPhraseQuery at lucene-misc-2.9.1.jar.
          This patch includes addition of a jar to trunk/lib directory.
          This jar has the corrected version of ComplexPhraseQueryParser.
          I couldn't include the corrected version of ComplexPhraseQueryParser in patch because it accesses a package-private field in Lucene's QueryParser.

          Show
          Ahmet Arslan added a comment - There is a fixed but forgotten bug in ComplexPhraseQuery at lucene-misc-2.9.1.jar. This patch includes addition of a jar to trunk/lib directory. This jar has the corrected version of ComplexPhraseQueryParser. I couldn't include the corrected version of ComplexPhraseQueryParser in patch because it accesses a package-private field in Lucene's QueryParser.
          Hide
          Ahmet Arslan added a comment -

          solr-core maven artifact already contains AbstractSolrTestCase, so it is deleted from test source directory.

          Show
          Ahmet Arslan added a comment - solr-core maven artifact already contains AbstractSolrTestCase, so it is deleted from test source directory.
          Hide
          Ahmet Arslan added a comment -

          To enable ComplexPhraseQueryParser in Solr 1.4.0 - Revisited

          Due to the reasons revealed here [1] this plugin should be loaded using the old way [2]

          [1] http://search-lucene.com/m/E49gN1naPyh
          [2] http://wiki.apache.org/solr/SolrPlugins#The_Old_Way

          1-) extract ComplexPhrase.zip and run 'mvn package'
          2-) unzip apache-solr-1.4.0.zip and copy ComplexPhrase/target/ComplexPhrase-1.0.jar to apache-solr-1.4.0/lib directory.
          3-) create a new apache-solr-1.4.0\dist\apache-solr-1.4.1-dev.war (by running 'ant dist') and use it.

          4-) register queryparser to solrhome/conf/solrconfig.xml by adding
          <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" />
          5-) enable it by appending &defType=complexphrase to search url.
          6-) Alternatively you can add

          {!complexphrase} in front of your query string. e.g. &q={!complexphrase}

          "s* b*"
          7-) More permanent usage can be configured in solrconfig.xml
          <requestHandler name="standard" class="solr.StandardRequestHandler" default="true">
          <lst name="defaults">
          <str name="defType">complexphrase</str>
          </lst>
          </requestHandler>

          Show
          Ahmet Arslan added a comment - To enable ComplexPhraseQueryParser in Solr 1.4.0 - Revisited Due to the reasons revealed here [1] this plugin should be loaded using the old way [2] [1] http://search-lucene.com/m/E49gN1naPyh [2] http://wiki.apache.org/solr/SolrPlugins#The_Old_Way 1-) extract ComplexPhrase.zip and run 'mvn package' 2-) unzip apache-solr-1.4.0.zip and copy ComplexPhrase/target/ComplexPhrase-1.0.jar to apache-solr-1.4.0/lib directory. 3-) create a new apache-solr-1.4.0\dist\apache-solr-1.4.1-dev.war (by running 'ant dist') and use it. 4-) register queryparser to solrhome/conf/solrconfig.xml by adding <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" /> 5-) enable it by appending &defType=complexphrase to search url. 6-) Alternatively you can add {!complexphrase} in front of your query string. e.g. &q={!complexphrase} "s* b*" 7-) More permanent usage can be configured in solrconfig.xml <requestHandler name="standard" class="solr.StandardRequestHandler" default="true"> <lst name="defaults"> <str name="defType">complexphrase</str> </lst> </requestHandler>
          Hide
          Hoss Man added a comment -

          Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email...

          http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E

          Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed.

          A unique token for finding these 240 issues in the future: hossversioncleanup20100527

          Show
          Hoss Man added a comment - Bulk updating 240 Solr issues to set the Fix Version to "next" per the process outlined in this email... http://mail-archives.apache.org/mod_mbox/lucene-dev/201005.mbox/%3Calpine.DEB.1.10.1005251052040.24672@radix.cryptio.net%3E Selection criteria was "Unresolved" with a Fix Version of 1.5, 1.6, 3.1, or 4.0. email notifications were suppressed. A unique token for finding these 240 issues in the future: hossversioncleanup20100527
          Hide
          Ahmet Arslan added a comment -

          There is a need for un-ordered proximity search. http://search-lucene.com/m/3W9fj2yzNy82/
          Configurable inOrder parameter is added where default behavior is true .

          The configuration below can be used to obtain same behavior of PhraseQuery in which order of terms is not important.

           
          <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin">
              <bool name="inOrder">false</bool>
          </queryParser>
          
          Show
          Ahmet Arslan added a comment - There is a need for un-ordered proximity search. http://search-lucene.com/m/3W9fj2yzNy82/ Configurable inOrder parameter is added where default behavior is true . The configuration below can be used to obtain same behavior of PhraseQuery in which order of terms is not important. <queryParser name= "complexphrase" class= "org.apache.solr.search.ComplexPhraseQParserPlugin" > <bool name= "inOrder" > false </bool> </queryParser>
          Hide
          Ahmet Arslan added a comment -

          Hyphen inside the phrase causes exception. e.g. "sulfur-reducing bacteria"
          Terje Eggestad's fix is added.

          Show
          Ahmet Arslan added a comment - Hyphen inside the phrase causes exception. e.g. "sulfur-reducing bacteria" Terje Eggestad's fix is added.
          Hide
          Salman Akram added a comment - - edited

          I integrated the patch and its working fine however, there were couple of issues.

          One is related to un-ordered proximity which seems to be fixed with the inOrder parameter but its not working for me (doesn't give any error but its still ordered). I will try to get the patch again coz I also merged it in early Nov so maybe it was applied after that.

          The other issue is that although proximity search works with phrases BUT its not very accurate e.g. If I want to search "a b" within 10 words of "c" the query would end up being "a b c"~10 but this will also return cases where "a" is not necessarily together with "b". Any scenario where these 3 words are within 10 words of each other will match.

          Is it possible in SOLR to do what I mentioned above? Any other patch? Something like " "a b" c "~10...

          Note: I was going through Lucene-1486 and there Ahmet mentioned that "Specifically : "(john johathon) smith"~10 " works perfectly. For me it seems there is no difference if I put the parenthesis or not.

          Thanks!

          Show
          Salman Akram added a comment - - edited I integrated the patch and its working fine however, there were couple of issues. One is related to un-ordered proximity which seems to be fixed with the inOrder parameter but its not working for me (doesn't give any error but its still ordered). I will try to get the patch again coz I also merged it in early Nov so maybe it was applied after that. The other issue is that although proximity search works with phrases BUT its not very accurate e.g. If I want to search "a b" within 10 words of "c" the query would end up being "a b c"~10 but this will also return cases where "a" is not necessarily together with "b". Any scenario where these 3 words are within 10 words of each other will match. Is it possible in SOLR to do what I mentioned above? Any other patch? Something like " "a b" c "~10... Note: I was going through Lucene-1486 and there Ahmet mentioned that "Specifically : "(john johathon) smith"~10 " works perfectly. For me it seems there is no difference if I put the parenthesis or not. Thanks!
          Hide
          Ahmet Arslan added a comment -

          Salman, what you are after is nested proximity search which is not currently available in Solr. For alternatives see http://search-lucene.com/m/94ONm1KRuAv1/

          Regarding parenthesis inside quotes and inOrder, they are covered in TestCases.

          What version of solr did you use?

          Show
          Ahmet Arslan added a comment - Salman, what you are after is nested proximity search which is not currently available in Solr. For alternatives see http://search-lucene.com/m/94ONm1KRuAv1/ Regarding parenthesis inside quotes and inOrder, they are covered in TestCases. What version of solr did you use?
          Hide
          Salman Akram added a comment -

          I am using SOLR 1.4.1 but integrated this patch in early Nov so maybe you committed the inOrder parameter after that?

          When you say "Regarding parenthesis inside quotes..." if this works and groups the words in phrase together won't it work for my case e.g. "(a b) c"~10?

          I guess if SurroundQuery doesn't use any analyzer it would be very difficult to make the existing queries work (I am using Standard Analyzer).

          Show
          Salman Akram added a comment - I am using SOLR 1.4.1 but integrated this patch in early Nov so maybe you committed the inOrder parameter after that? When you say "Regarding parenthesis inside quotes..." if this works and groups the words in phrase together won't it work for my case e.g. "(a b) c"~10? I guess if SurroundQuery doesn't use any analyzer it would be very difficult to make the existing queries work (I am using Standard Analyzer).
          Hide
          Ahmet Arslan added a comment -

          Use the most recent file which is non-gray color. Also there is a "date attached" info for files.
          It works for "(a b) c"~10. This is equivalent to "a c"~10 OR "b c"~10.

          SurroundQueryParser does not use Analyzer. It recommended to heavily use wildcard operator instead.
          e.g. instead of searching foo bar, you search foo* bar*

          But if you are using Standard Analyzer which does not have stemming in it, I think you can use Surround.
          You can pre-lowercase your queries etc. You can even pre-analyze your queries since your analyzer does not inject new tokens.
          But your queries must be well formed, there is not default operator in this.

          But I think it is better to discuss these things in solr/lucene-user mailing list.

          Show
          Ahmet Arslan added a comment - Use the most recent file which is non-gray color. Also there is a "date attached" info for files. It works for "(a b) c"~10. This is equivalent to "a c"~10 OR "b c"~10. SurroundQueryParser does not use Analyzer. It recommended to heavily use wildcard operator instead. e.g. instead of searching foo bar, you search foo* bar* But if you are using Standard Analyzer which does not have stemming in it, I think you can use Surround. You can pre-lowercase your queries etc. You can even pre-analyze your queries since your analyzer does not inject new tokens. But your queries must be well formed, there is not default operator in this. But I think it is better to discuss these things in solr/lucene-user mailing list.
          Hide
          Salman Akram added a comment -

          I am trying to use CommonGrams with this patch but doesn't seem to work.

          If I don't add

          {!complexphrase} it uses CommonGramsQueryFilterFactory and proper bi-grams are made but of course doesn't use this patch.

          If I add {!complexphrase}

          it simply does it the old way i.e. ignore CommonGrams.

          Can you please help how can I combine both these features?

          Show
          Salman Akram added a comment - I am trying to use CommonGrams with this patch but doesn't seem to work. If I don't add {!complexphrase} it uses CommonGramsQueryFilterFactory and proper bi-grams are made but of course doesn't use this patch. If I add {!complexphrase} it simply does it the old way i.e. ignore CommonGrams. Can you please help how can I combine both these features?
          Hide
          Salman Akram added a comment -

          I tried the patch with latest non-grayed file but still inOrder doesn't seem to have any impact.

          Results for "a b"~5 and "b a"~5 are still different.

          Also any feedback about CommonGrams integration?

          Thanks a lot for all the help!

          Show
          Salman Akram added a comment - I tried the patch with latest non-grayed file but still inOrder doesn't seem to have any impact. Results for "a b"~5 and "b a"~5 are still different. Also any feedback about CommonGrams integration? Thanks a lot for all the help!
          Hide
          Ahmet Arslan added a comment -

          When you add &debugQuery=on to your search URL, you should see something like in the debug section:
          spanNear([text:a, text:b], 5, false) , false here means un-ordered phrase query. Do you see it?

          I will look into CommonGrams this weekend.

          Show
          Ahmet Arslan added a comment - When you add &debugQuery=on to your search URL, you should see something like in the debug section: spanNear( [text:a, text:b] , 5, false) , false here means un-ordered phrase query. Do you see it? I will look into CommonGrams this weekend.
          Hide
          Ahsan Iqbal added a comment -

          Hi

          I am having same issue (inOrder does not seem to work) with

          {!complexphrase}

          , in the debug section it shows spanNear([text:a, text:b], 5, true),
          my solrconfig.xml looks like

          <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" >
          <bool name="inOrder">false</bool>
          </queryParser>

          Show
          Ahsan Iqbal added a comment - Hi I am having same issue (inOrder does not seem to work) with {!complexphrase} , in the debug section it shows spanNear( [text:a, text:b] , 5, true), my solrconfig.xml looks like <queryParser name="complexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin" > <bool name="inOrder">false</bool> </queryParser>
          Hide
          Ahsan Iqbal added a comment -

          Hi Ahmet

          I have read on mailing lists dated "Mon, 20 Jul 2009" that you had merged the surround query parser with solr.
          I have tried that by downloading jar file for surround query parser. and then pasting that jar file in web-inf/lib
          and configured query parser in solrconfig.xml as
          <queryParser name="SurroundQParser" class="org.apache.lucene.queryParser.surround.parser.QueryParser"/>

          then web page the following exception comes

          org.apache.solr.common.SolrException: Error Instantiating QParserPlugin, org.apache.lucene.queryParser.surround.parser.QueryParser is not a org.apache.solr.search.QParserPlugin

          can u guide what i m doing wrong....

          Regards
          Ahsan

          Show
          Ahsan Iqbal added a comment - Hi Ahmet I have read on mailing lists dated "Mon, 20 Jul 2009" that you had merged the surround query parser with solr. I have tried that by downloading jar file for surround query parser. and then pasting that jar file in web-inf/lib and configured query parser in solrconfig.xml as <queryParser name="SurroundQParser" class="org.apache.lucene.queryParser.surround.parser.QueryParser"/> then web page the following exception comes org.apache.solr.common.SolrException: Error Instantiating QParserPlugin, org.apache.lucene.queryParser.surround.parser.QueryParser is not a org.apache.solr.search.QParserPlugin can u guide what i m doing wrong.... Regards Ahsan
          Hide
          Salman Akram added a comment -

          Ahmet,

          I would be waiting for your response on CommonGrams. Would be grateful if you can look into it this weekend. Thanks!

          Show
          Salman Akram added a comment - Ahmet, I would be waiting for your response on CommonGrams. Would be grateful if you can look into it this weekend. Thanks!
          Hide
          Salman Akram added a comment -

          Although I would be asking this question on the mailing list as well but since its related to this patch so I wanted to check if this patch would work fine with SurroundQueryParser or if Surround does it itself? This patch functionality is really important for me.

          Thanks a lot!

          Show
          Salman Akram added a comment - Although I would be asking this question on the mailing list as well but since its related to this patch so I wanted to check if this patch would work fine with SurroundQueryParser or if Surround does it itself? This patch functionality is really important for me. Thanks a lot!
          Hide
          Salman Akram added a comment -

          Any updates on integration with CommonGrams? Thanks

          Show
          Salman Akram added a comment - Any updates on integration with CommonGrams? Thanks
          Hide
          Salman Akram added a comment -

          Reminder! Any updates regarding integration with CommonGrams? Thanks

          Show
          Salman Akram added a comment - Reminder! Any updates regarding integration with CommonGrams? Thanks
          Hide
          Robert Muir added a comment -

          Bulk move 3.2 -> 3.3

          Show
          Robert Muir added a comment - Bulk move 3.2 -> 3.3
          Hide
          Thomas Fischer added a comment -

          Will the complexphrase search works fine with e.g. GOK:"PXB 80?", it will throw an exception if there is no space present, e.g. GOK:"PXB80?". The exception is:
          Unknown query type "org.apache.lucene.search.WildcardQuery" found in phrase query string "PXB80?"

          Show
          Thomas Fischer added a comment - Will the complexphrase search works fine with e.g. GOK:"PXB 80?", it will throw an exception if there is no space present, e.g. GOK:"PXB80?". The exception is: Unknown query type "org.apache.lucene.search.WildcardQuery" found in phrase query string "PXB80?"
          Hide
          Ahmet Arslan added a comment -

          Update for solr 3.3.0

          • Download apache-solr-3.3.0-src.tgz
          • Download most latest ComplexPhrase.zip
          • 'mvn package' will generate 3 files under target folder
            copy them to apache-solr-3.3.0/solr/lib/
            • cp target/ComplexPhrase-* Downloads/apache-solr-3.3.0/solr/lib/
          • call 'ant clean dist' to create a new apache-solr-3.3-SNAPSHOT.war file under solr/dist folder
          Show
          Ahmet Arslan added a comment - Update for solr 3.3.0 Download apache-solr-3.3.0-src.tgz Download most latest ComplexPhrase.zip 'mvn package' will generate 3 files under target folder copy them to apache-solr-3.3.0/solr/lib/ cp target/ComplexPhrase-* Downloads/apache-solr-3.3.0/solr/lib/ call 'ant clean dist' to create a new apache-solr-3.3-SNAPSHOT.war file under solr/dist folder
          Hide
          Robert Muir added a comment -

          3.4 -> 3.5

          Show
          Robert Muir added a comment - 3.4 -> 3.5
          Hide
          meghana added a comment -

          I have installed complexphrase parser and followed all instructions for installation... when i run with defType=complexphrase , i get below error.
          HTTP Status 500 - luceneMatchVersion java.lang.NoSuchFieldError: luceneMatchVersion at org.apache.solr.search.SolrComplexPhraseQueryParser

          how to solve this. i am new to solr

          Show
          meghana added a comment - I have installed complexphrase parser and followed all instructions for installation... when i run with defType=complexphrase , i get below error. HTTP Status 500 - luceneMatchVersion java.lang.NoSuchFieldError: luceneMatchVersion at org.apache.solr.search.SolrComplexPhraseQueryParser how to solve this. i am new to solr
          Hide
          Tomás Fernández Löbbe added a comment -

          The class org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser was committed and it's available in "modules/queryparser". How should this patch proceed in order to make that QParser available in Solr? Should it be added to the trunk? as a contrib?

          Show
          Tomás Fernández Löbbe added a comment - The class org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser was committed and it's available in "modules/queryparser". How should this patch proceed in order to make that QParser available in Solr? Should it be added to the trunk? as a contrib?
          Hide
          Ahmet Arslan added a comment -

          Committed o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser does not work with non-default fields. Several Lucene users raised this issue on mailing lists. Mark Harwood said the following on LUCENE-1486 which is still unresolved. However it didn't get any attention.

          Fixing this would require changing the package name of ComplexPhraseQueryParser or changing the visibility of "field" in the QueryParser base class to "protected".
          Anyone have any strong feelings about which of these is the most acceptable?

          That's why attachment of this issue is not consuming committed o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser and released as a solr-plugin.

          Show
          Ahmet Arslan added a comment - Committed o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser does not work with non-default fields. Several Lucene users raised this issue on mailing lists. Mark Harwood said the following on LUCENE-1486 which is still unresolved. However it didn't get any attention. Fixing this would require changing the package name of ComplexPhraseQueryParser or changing the visibility of "field" in the QueryParser base class to "protected". Anyone have any strong feelings about which of these is the most acceptable? That's why attachment of this issue is not consuming committed o.a.l.queryparser.complexPhrase.ComplexPhraseQueryParser and released as a solr-plugin.
          Hide
          Tomás Fernández Löbbe added a comment -

          That's true, but probably this Jira should focus in using the available ComplexPhraseQueryParser and there should be another Jira (a Lucene one) to fix that problem. The parser was moved for a reason and if we use a duplicated version of the parser in a different package (the old one) this will never be committed.

          Show
          Tomás Fernández Löbbe added a comment - That's true, but probably this Jira should focus in using the available ComplexPhraseQueryParser and there should be another Jira (a Lucene one) to fix that problem. The parser was moved for a reason and if we use a duplicated version of the parser in a different package (the old one) this will never be committed.
          Hide
          Ahmet Arslan added a comment -

          I imagined that LUCENE-1486 will be closed/fixed in the future, hopefully including the non default field patch. Are you saying that non default field problem should be handled in a separate issue (other than LUCENE-1486) ?

          Show
          Ahmet Arslan added a comment - I imagined that LUCENE-1486 will be closed/fixed in the future, hopefully including the non default field patch. Are you saying that non default field problem should be handled in a separate issue (other than LUCENE-1486 ) ?
          Hide
          Tomás Fernández Löbbe added a comment -

          It might be LUCENE-1486 or a new Lucene Jira, but what I say is that it should not be fixed in this issue. What I'm proposing is to only provide the necessary classes in Solr to be able to work with the current version of ComplexPhraseQueryParser, even if it is limited in some aspects. Once the ComplexPhraseQueryParser is improved we'll get those improvements in Solr (either automatically or by improving something from this side).

          Show
          Tomás Fernández Löbbe added a comment - It might be LUCENE-1486 or a new Lucene Jira, but what I say is that it should not be fixed in this issue. What I'm proposing is to only provide the necessary classes in Solr to be able to work with the current version of ComplexPhraseQueryParser, even if it is limited in some aspects. Once the ComplexPhraseQueryParser is improved we'll get those improvements in Solr (either automatically or by improving something from this side).
          Hide
          Ahmet Arslan added a comment -

          path for trunk

          Show
          Ahmet Arslan added a comment - path for trunk
          Hide
          Ahmet Arslan added a comment -

          to make inOrder parameter configurable

          Show
          Ahmet Arslan added a comment - to make inOrder parameter configurable
          Hide
          Ahmet Arslan added a comment -

          to enable non-default field searches

          Show
          Ahmet Arslan added a comment - to enable non-default field searches
          Hide
          Tomás Fernández Löbbe added a comment -

          In my opinion, we should not be relying on un-committed Lucene code, as it may change or even never get committed. I think this Jira should only consider the current ComplexPhraseQueryParser and we can create a new Solr Jira that depends on LUCENE-3758 for the order/unordered query.

          Show
          Tomás Fernández Löbbe added a comment - In my opinion, we should not be relying on un-committed Lucene code, as it may change or even never get committed. I think this Jira should only consider the current ComplexPhraseQueryParser and we can create a new Solr Jira that depends on LUCENE-3758 for the order/unordered query.
          Hide
          Tomás Fernández Löbbe added a comment -

          I'm attaching an alternative patch with what I propose:
          It works fine with what's committed today in Lucene, it doesn't require LUCENE-3758 committed to compile or LUCENE-1486 committed to run the tests, however, it
          has the limitations of the currently committed ComplexPhraseQueryParser.
          I have some commented testUnorderedPhraseQuery. It is commented because it currently doesn't run to the the limitations of LUCENE-3758
          testFieldedQuery is also commented, as it doesn't run until LUCENE-1486 is committed.
          I added testAutogeneratingPhraseQueries, but it is also commented as hyphenated queries don't work when "autogenerate phrase queries" is set to true. This is something we should continue working on in this issue.
          I would like to hear opinions on how to continue working.

          Show
          Tomás Fernández Löbbe added a comment - I'm attaching an alternative patch with what I propose: It works fine with what's committed today in Lucene, it doesn't require LUCENE-3758 committed to compile or LUCENE-1486 committed to run the tests, however, it has the limitations of the currently committed ComplexPhraseQueryParser. I have some commented testUnorderedPhraseQuery. It is commented because it currently doesn't run to the the limitations of LUCENE-3758 testFieldedQuery is also commented, as it doesn't run until LUCENE-1486 is committed. I added testAutogeneratingPhraseQueries, but it is also commented as hyphenated queries don't work when "autogenerate phrase queries" is set to true. This is something we should continue working on in this issue. I would like to hear opinions on how to continue working.
          Hide
          Tomás Fernández Löbbe added a comment -

          Did anybody got the chance to see these two patches? are there any more opinions on this?

          Show
          Tomás Fernández Löbbe added a comment - Did anybody got the chance to see these two patches? are there any more opinions on this?
          Hide
          Hoss Man added a comment -

          Bulk changing fixVersion 3.6 to 4.0 for any open issues that are unassigned and have not been updated since March 19.

          Email spam suppressed for this bulk edit; search for hoss20120323nofix36 to identify all issues edited

          Show
          Hoss Man added a comment - Bulk changing fixVersion 3.6 to 4.0 for any open issues that are unassigned and have not been updated since March 19. Email spam suppressed for this bulk edit; search for hoss20120323nofix36 to identify all issues edited
          Hide
          Madhusudan added a comment -

          I am upgrading Solr to version 3.6 from version 1.4.1 and applied this patch. The Solr is built successfully. After indexing some document I performed a wildcard phrase query "weekly repo*", no documents were returned. However, by appending "&defType=complexphrase" the solr returned the document containing "weekly report". It seems that the complex phrase is not taken as default search handler.
          Also, when restarting solr I discovered following error in log file as:

          SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.search.ComplexPhraseQParserPlugin'
          at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:394)
          at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:419)
          at org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:441)
          at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1612)
          at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1606)
          at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1639)
          at org.apache.solr.core.SolrCore.initQParsers(SolrCore.java:1556)
          at org.apache.solr.core.SolrCore.<init>(SolrCore.java:555)
          at org.apache.solr.core.CoreContainer.create(CoreContainer.java:483)
          at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335)
          at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219)
          at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161)
          at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96)
          at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
          at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
          at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713)
          at org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
          at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
          at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
          at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
          at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
          at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
          at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156)
          at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
          at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152)
          at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
          at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
          at org.mortbay.jetty.Server.doStart(Server.java:224)
          at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
          at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
          at java.lang.reflect.Method.invoke(Unknown Source)
          at org.mortbay.start.Main.invokeMain(Main.java:194)
          at org.mortbay.start.Main.start(Main.java:534)
          at org.mortbay.start.Main.start(Main.java:441)
          at org.mortbay.start.Main.main(Main.java:119)
          Caused by: java.lang.ClassNotFoundException: org.apache.solr.search.ComplexPhraseQParserPlugin
          at java.net.URLClassLoader$1.run(Unknown Source)
          at java.security.AccessController.doPrivileged(Native Method)
          at java.net.URLClassLoader.findClass(Unknown Source)
          at java.lang.ClassLoader.loadClass(Unknown Source)
          at java.net.FactoryURLClassLoader.loadClass(Unknown Source)
          at java.lang.ClassLoader.loadClass(Unknown Source)
          at java.lang.Class.forName0(Native Method)
          at java.lang.Class.forName(Unknown Source)
          at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:378)
          ... 37 more

          I would like some help regarding the use of this patch in new solr.

          Show
          Madhusudan added a comment - I am upgrading Solr to version 3.6 from version 1.4.1 and applied this patch. The Solr is built successfully. After indexing some document I performed a wildcard phrase query "weekly repo*", no documents were returned. However, by appending "&defType=complexphrase" the solr returned the document containing "weekly report". It seems that the complex phrase is not taken as default search handler. Also, when restarting solr I discovered following error in log file as: SEVERE: org.apache.solr.common.SolrException: Error loading class 'org.apache.solr.search.ComplexPhraseQParserPlugin' at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:394) at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:419) at org.apache.solr.core.SolrCore.createInitInstance(SolrCore.java:441) at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1612) at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1606) at org.apache.solr.core.SolrCore.initPlugins(SolrCore.java:1639) at org.apache.solr.core.SolrCore.initQParsers(SolrCore.java:1556) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:555) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:483) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:335) at org.apache.solr.core.CoreContainer.load(CoreContainer.java:219) at org.apache.solr.core.CoreContainer$Initializer.initialize(CoreContainer.java:161) at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:96) at org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:713) at org.mortbay.jetty.servlet.Context.startContext(Context.java:140) at org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282) at org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518) at org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.jetty.handler.ContextHandlerCollection.doStart(ContextHandlerCollection.java:156) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerCollection.doStart(HandlerCollection.java:152) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130) at org.mortbay.jetty.Server.doStart(Server.java:224) at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50) at org.mortbay.xml.XmlConfiguration.main(XmlConfiguration.java:985) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.mortbay.start.Main.invokeMain(Main.java:194) at org.mortbay.start.Main.start(Main.java:534) at org.mortbay.start.Main.start(Main.java:441) at org.mortbay.start.Main.main(Main.java:119) Caused by: java.lang.ClassNotFoundException: org.apache.solr.search.ComplexPhraseQParserPlugin at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.net.FactoryURLClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Unknown Source) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:378) ... 37 more I would like some help regarding the use of this patch in new solr.
          Hide
          Mike Bria added a comment - - edited

          Hi everyone,

          Sorry, but I'm green to this stuff. How do I "apply/install" a "*.patch" file?

          I downloaded and successfully built (well, "packaged", via mvn) the ComplexPhrase.zip from Jul-2011.

          I then downloaded the 'SOLR-1604-alternative.patch' from Feb-2012. I can open and view it via a text editor...but I have no idea what to do to "apply" it? I'm working on a RH Linux box at the moment.

          Can anyone guide me and/or point me in the right direction please?

          Thanks!
          Mike

          Show
          Mike Bria added a comment - - edited Hi everyone, Sorry, but I'm green to this stuff. How do I "apply/install" a "*.patch" file? I downloaded and successfully built (well, "packaged", via mvn) the ComplexPhrase.zip from Jul-2011. I then downloaded the ' SOLR-1604 -alternative.patch' from Feb-2012. I can open and view it via a text editor...but I have no idea what to do to "apply" it? I'm working on a RH Linux box at the moment. Can anyone guide me and/or point me in the right direction please? Thanks! Mike
          Hide
          Ahmet Arslan added a comment -

          There are two separate ways to enable this functionality.

          • You can consume *.zip attachments as a solr plugin. Which does not require source code modification, but this particular case requires re-creating solr.war. http://wiki.apache.org/solr/SolrPlugins
          Show
          Ahmet Arslan added a comment - There are two separate ways to enable this functionality. You can consume *.zip attachments as a solr plugin. Which does not require source code modification, but this particular case requires re-creating solr.war. http://wiki.apache.org/solr/SolrPlugins *.patch files contains source code modifications. http://wiki.apache.org/solr/HowToContribute#Working_With_Patches
          Hide
          Hoss Man added a comment -

          bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment

          Show
          Hoss Man added a comment - bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
          Hide
          Robert Muir added a comment -

          rmuir20120906-bulk-40-change

          Show
          Robert Muir added a comment - rmuir20120906-bulk-40-change
          Hide
          Phani Vempaty added a comment -

          Hello all,

          Could I get the grammar file (.jj file) for ComplexPhrase one - It's not there as part of the patch/zip file.

          Thanks,
          Phani.

          Show
          Phani Vempaty added a comment - Hello all, Could I get the grammar file (.jj file) for ComplexPhrase one - It's not there as part of the patch/zip file. Thanks, Phani.
          Hide
          Ahmet Arslan added a comment -

          Could I get the grammar file (.jj file) for ComplexPhrase one - It's not there as part of the patch/zip file.

          It does not have a separate grammar file. It just extends QueryPaser.

          Show
          Ahmet Arslan added a comment - Could I get the grammar file (.jj file) for ComplexPhrase one - It's not there as part of the patch/zip file. It does not have a separate grammar file. It just extends QueryPaser.
          Hide
          Hoss Man added a comment -

          removing 4.0 from fix version since this depends on LUCENE-1486 which is not already in the 4x branch .. can revist if there is movement on LUCENE-1486

          Show
          Hoss Man added a comment - removing 4.0 from fix version since this depends on LUCENE-1486 which is not already in the 4x branch .. can revist if there is movement on LUCENE-1486
          Hide
          Jun Wang added a comment -

          I have noticed that LUCENE-1486 will be fixed in v4.1, and I have found that this patch can not work with solr 4.0. Is there new patch for 4.0?

          Show
          Jun Wang added a comment - I have noticed that LUCENE-1486 will be fixed in v4.1, and I have found that this patch can not work with solr 4.0. Is there new patch for 4.0?
          Hide
          Uwe Schindler added a comment -

          If there would be a working patch, it could be in 4.0. It was just moved out because nobody took care. It's your turn!

          Show
          Uwe Schindler added a comment - If there would be a working patch, it could be in 4.0. It was just moved out because nobody took care. It's your turn!
          Hide
          Ahmet Arslan added a comment -

          this supposed to work with solr 4.0

          Show
          Ahmet Arslan added a comment - this supposed to work with solr 4.0
          Hide
          Jun Wang added a comment -

          Hi, Arslan, when using your patch for 4.0, I got a Exception: java.lang.IllegalArgumentException: Unknown query type "org.apache.lucene.search.PrefixQuery" found in phrase query string "mp*", my query is q=subject:"mp*"&wt=json. This exception was thrown at QueryComponent#process.

          Show
          Jun Wang added a comment - Hi, Arslan, when using your patch for 4.0, I got a Exception: java.lang.IllegalArgumentException: Unknown query type "org.apache.lucene.search.PrefixQuery" found in phrase query string "mp*", my query is q=subject:"mp*"&wt=json. This exception was thrown at QueryComponent#process.
          Hide
          Ahmet Arslan added a comment -

          Jun Wang query string "mp*" is somehow invalid. Use &q=mp* instead. This parser expects at least two clauses inside quotes.

          Show
          Ahmet Arslan added a comment - Jun Wang query string "mp*" is somehow invalid. Use &q=mp* instead. This parser expects at least two clauses inside quotes.
          Hide
          Ahmet Arslan added a comment -

          Includes README.txt that contain instruction for Solr 4.0.0

          Show
          Ahmet Arslan added a comment - Includes README.txt that contain instruction for Solr 4.0.0
          Hide
          Roman Slavik added a comment -

          Hi, I downloaded last version of ComplexPhrase (24/Oct/12 02:30) but have problem with junit test. Here is error log :

          test(org.apache.solr.search.ComplexPhraseQParserPluginTest)  Time elapsed: 0.191 sec  <<< ERROR!
          java.lang.RuntimeException: Exception during query
          	at __randomizedtesting.SeedInfo.seed([4BF35CC9C13F3B15:C3A763136FC356ED]:0)
          	at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:283)
          	at org.apache.solr.search.ComplexPhraseQParserPluginTest.test(ComplexPhraseQParserPluginTest.java:158)
          	// nothing interest here
          Caused by: java.lang.IllegalArgumentException: Unknown query type "org.apache.lucene.search.ConstantScoreQuery" found in phrase query string "jo* [sma TO smz]"
          	at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser$ComplexPhraseQuery.rewrite(ComplexPhraseQueryParser.java:297)
          	at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:599)
          	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:646)
          	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:280)
          	at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1385)
          	at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1260)
          	at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:390)
          	at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:411)
          	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:206)
          	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
          	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
          	at org.apache.solr.util.TestHarness.query(TestHarness.java:364)
          	at org.apache.solr.util.TestHarness.query(TestHarness.java:346)
          	at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:273)
          	... 41 more
          

          Is it error on my side (I didn't change anything)? Or some crucial error?

          Show
          Roman Slavik added a comment - Hi, I downloaded last version of ComplexPhrase (24/Oct/12 02:30) but have problem with junit test. Here is error log : test(org.apache.solr.search.ComplexPhraseQParserPluginTest) Time elapsed: 0.191 sec <<< ERROR! java.lang.RuntimeException: Exception during query at __randomizedtesting.SeedInfo.seed([4BF35CC9C13F3B15:C3A763136FC356ED]:0) at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:283) at org.apache.solr.search.ComplexPhraseQParserPluginTest.test(ComplexPhraseQParserPluginTest.java:158) // nothing interest here Caused by: java.lang.IllegalArgumentException: Unknown query type "org.apache.lucene.search.ConstantScoreQuery" found in phrase query string "jo* [sma TO smz]" at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser$ComplexPhraseQuery.rewrite(ComplexPhraseQueryParser.java:297) at org.apache.lucene.search.IndexSearcher.rewrite(IndexSearcher.java:599) at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:646) at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:280) at org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1385) at org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1260) at org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:390) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:411) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:206) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699) at org.apache.solr.util.TestHarness.query(TestHarness.java:364) at org.apache.solr.util.TestHarness.query(TestHarness.java:346) at org.apache.solr.util.AbstractSolrTestCase.assertQ(AbstractSolrTestCase.java:273) ... 41 more Is it error on my side (I didn't change anything)? Or some crucial error?
          Hide
          Ryan Long added a comment -

          I also see the error Roman posted, with the latest version

          Show
          Ryan Long added a comment - I also see the error Roman posted, with the latest version
          Hide
          Dmitry Kan added a comment -

          This is ComplexPhrase project based on the version submitted on 21/Jul/11. It compiles and runs under solr 3.4. I have uncommented the tests in /org/apache/solr/search/ComplexPhraseQParserPluginTest.java and they passed.

          Show
          Dmitry Kan added a comment - This is ComplexPhrase project based on the version submitted on 21/Jul/11. It compiles and runs under solr 3.4. I have uncommented the tests in /org/apache/solr/search/ComplexPhraseQParserPluginTest.java and they passed.
          Hide
          Dmitry Kan added a comment -

          Hello! Great work!

          I have two questions:

          1) What would it take to incorporate phrase searches into this extended query parser?
          "\"a b\" c"~100
          that is, "a b" (phrase search) is found in that order and exactly side by side <=100 tokens away from c.

          2) does this implementation support the Boolean operators, like AND, OR, NOT (at least OR and NOT are supported as far as I can see)? Can they be nested?

          Show
          Dmitry Kan added a comment - Hello! Great work! I have two questions: 1) What would it take to incorporate phrase searches into this extended query parser? "\"a b\" c"~100 that is, "a b" (phrase search) is found in that order and exactly side by side <=100 tokens away from c. 2) does this implementation support the Boolean operators, like AND, OR, NOT (at least OR and NOT are supported as far as I can see)? Can they be nested?
          Hide
          Paul Morris added a comment -

          Hi Ahmet and fellow ComplexPhrase users,

          We have been using the ComplexPhraseQueryParser for a number of years with our SOLR installation, but we were hoping to upgrade to SOLR 4.1 and I was just wondering if ComplexPhrase works with the latest SOLR release?

          We are still on SOLR 1.4.1 so keen to take advantage of all that SOLR 4.1 can offer, but we can't go there without ComplexPhrase

          Much thanks if anyone has anyone has any thoughts.

          Show
          Paul Morris added a comment - Hi Ahmet and fellow ComplexPhrase users, We have been using the ComplexPhraseQueryParser for a number of years with our SOLR installation, but we were hoping to upgrade to SOLR 4.1 and I was just wondering if ComplexPhrase works with the latest SOLR release? We are still on SOLR 1.4.1 so keen to take advantage of all that SOLR 4.1 can offer, but we can't go there without ComplexPhrase Much thanks if anyone has anyone has any thoughts.
          Hide
          Ahmet Arslan added a comment -

          Bring to 4.2.1 version

          Show
          Ahmet Arslan added a comment - Bring to 4.2.1 version
          Hide
          ettey siva added a comment -

          Hi Ahmet
          I have installed complexphrase parser and followed all instructions for installation...
          Everything is fine except getting NULL Pointer exception when I start JBOSS.

          19:46:03,909 ERROR [SolrCore] java.lang.NullPointerException
          at org.apache.solr.schema.IndexSchema$DynamicReplacement$DynamicPattern$NameEndsWith.matches(IndexSchema.java:960)
          at org.apache.solr.schema.IndexSchema$DynamicReplacement.matches(IndexSchema.java:974)
          at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1221)
          at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:418)
          at org.apache.lucene.analysis.AnalyzerWrapper.initReader(AnalyzerWrapper.java:81)
          at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175)
          at org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:507)
          at org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:495)
          at org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1109)
          at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358)
          at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257)
          at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181)
          at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170)
          at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:121)
          at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.parse(ComplexPhraseQueryParser.java:107)
          at org.apache.solr.search.ComplexPhraseQParser.parse(ComplexPhraseQParserPlugin.java:108)
          at org.apache.solr.search.QParser.getQuery(QParser.java:142)
          at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:142)
          at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
          at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904)
          at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64)
          at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1693)
          at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
          at java.util.concurrent.FutureTask.run(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
          at java.lang.Thread.run(Unknown Source)

          Thanks in advance.

          Show
          ettey siva added a comment - Hi Ahmet I have installed complexphrase parser and followed all instructions for installation... Everything is fine except getting NULL Pointer exception when I start JBOSS. 19:46:03,909 ERROR [SolrCore] java.lang.NullPointerException at org.apache.solr.schema.IndexSchema$DynamicReplacement$DynamicPattern$NameEndsWith.matches(IndexSchema.java:960) at org.apache.solr.schema.IndexSchema$DynamicReplacement.matches(IndexSchema.java:974) at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1221) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:418) at org.apache.lucene.analysis.AnalyzerWrapper.initReader(AnalyzerWrapper.java:81) at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) at org.apache.lucene.queryparser.classic.QueryParserBase.newFieldQuery(QueryParserBase.java:507) at org.apache.lucene.queryparser.classic.QueryParserBase.getFieldQuery(QueryParserBase.java:495) at org.apache.lucene.queryparser.classic.QueryParserBase.handleBareTokenQuery(QueryParserBase.java:1109) at org.apache.lucene.queryparser.classic.QueryParser.Term(QueryParser.java:358) at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:257) at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:181) at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:170) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:121) at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.parse(ComplexPhraseQueryParser.java:107) at org.apache.solr.search.ComplexPhraseQParser.parse(ComplexPhraseQParserPlugin.java:108) at org.apache.solr.search.QParser.getQuery(QParser.java:142) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:142) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1693) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Thanks in advance.
          Hide
          Ahmet Arslan added a comment -

          Hi ettey siva, what version of solr did you use?

          Show
          Ahmet Arslan added a comment - Hi ettey siva , what version of solr did you use?
          Hide
          ettey siva added a comment -

          Hi Ahmet
          We are using solr 4.4.
          Issue was resolved by uncommenting <defaultSearchField>text</defaultSearchField> in schema.xml
          Thank you

          Show
          ettey siva added a comment - Hi Ahmet We are using solr 4.4. Issue was resolved by uncommenting <defaultSearchField>text</defaultSearchField> in schema.xml Thank you
          Hide
          Rebecca Tang added a comment -

          I have a question about upgrading solr with this plugin. Because the plugin has to be built into solr.war, I need to go through the process each time I upgrade solr. It's rather cumbersome. Does anyone have a better solution?

          Show
          Rebecca Tang added a comment - I have a question about upgrading solr with this plugin. Because the plugin has to be built into solr.war, I need to go through the process each time I upgrade solr. It's rather cumbersome. Does anyone have a better solution?
          Hide
          Dmitry Kan added a comment -

          Rebecca Tang you can define a solr core (even for a single index) and use its lib directory to copy the complex phrase parser jar.

          https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml

          HTH

          Show
          Dmitry Kan added a comment - Rebecca Tang you can define a solr core (even for a single index) and use its lib directory to copy the complex phrase parser jar. https://cwiki.apache.org/confluence/display/solr/Solr+Cores+and+solr.xml HTH
          Hide
          Ahmet Arslan added a comment -

          Rebecca, for the time being thats the only way to use it. Lib directory fails as explained here : http://search-lucene.com/m/E49gN1naPyh
          The problem is fielded queries do not work with current committed CPQP. If LUCENE-1486 gets fixed then this plugin can consume original CPQP.

          Show
          Ahmet Arslan added a comment - Rebecca, for the time being thats the only way to use it. Lib directory fails as explained here : http://search-lucene.com/m/E49gN1naPyh The problem is fielded queries do not work with current committed CPQP. If LUCENE-1486 gets fixed then this plugin can consume original CPQP.
          Hide
          suren added a comment -

          Ahmet,
          I am using solr 4.3.1. do i still need to apply this patch ? if yes please tell me what patch to apply, I see lot of patches here, not sure which patch for what version of solr.

          Show
          suren added a comment - Ahmet, I am using solr 4.3.1. do i still need to apply this patch ? if yes please tell me what patch to apply, I see lot of patches here, not sure which patch for what version of solr.
          Hide
          Randall Rosales Solorzano added a comment -

          Hi everyone,

          Does this plugin supports date searches?
          I noticed that doing something like this:
          q=Project.CommenceDate:[2006-08-01T00:00:00Z%20TO%202006-08-01T00:00:00Z]

          gives no results (When I use edismax it does return some results).
          I also tried:
          q=Project.CommenceDate:2006-08-01T00:00:00Z

          but I get this error:
          Invalid Date String:'2006-08-01T00'

          Thanks,
          Randall Rosales

          Show
          Randall Rosales Solorzano added a comment - Hi everyone, Does this plugin supports date searches? I noticed that doing something like this: q=Project.CommenceDate: [2006-08-01T00:00:00Z%20TO%202006-08-01T00:00:00Z] gives no results (When I use edismax it does return some results). I also tried: q=Project.CommenceDate:2006-08-01T00:00:00Z but I get this error: Invalid Date String:'2006-08-01T00' Thanks, Randall Rosales
          Hide
          Erick Erickson added a comment -

          Two things:

          1> dots in field names are risky, in general you should only use valid Java variable naming conventions.
          2> How are you submitting this? Because ':' is the delimiter between fields and values, you need to escape this line if you're submitting this via a browser, as 00\:00\:00Z

          BTW, you'll get faster responses to this kind of question if you raise it on the user's list, more eyes see it.

          Show
          Erick Erickson added a comment - Two things: 1> dots in field names are risky, in general you should only use valid Java variable naming conventions. 2> How are you submitting this? Because ':' is the delimiter between fields and values, you need to escape this line if you're submitting this via a browser, as 00\:00\:00Z BTW, you'll get faster responses to this kind of question if you raise it on the user's list, more eyes see it.
          Hide
          Anonymous added a comment -

          Hey,
          I have build Complex Phrase 4.2.1 and followed the steps given in first comment. I am using this parser with solr 4.6 version and when I executed URL by appending &defType=complexphrase I got an error given below. Any help would be appreciated thanks.
          java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/solr/search/SolrIndexReader at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:724) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:434) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.NoClassDefFoundError: org/apache/solr/search/SolrIndexReader at org.apache.solr.search.ComplexPhraseQParserPlugin.createParser(ComplexPhraseQParserPlugin.java:56) at org.apache.solr.search.QParser.getParser(QParser.java:347) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:141) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) ... 16 more Caused by: java.lang.ClassNotFoundException: org.apache.solr.search.SolrIndexReader at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.net.FactoryURLClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 24 more

          Show
          Anonymous added a comment - Hey, I have build Complex Phrase 4.2.1 and followed the steps given in first comment. I am using this parser with solr 4.6 version and when I executed URL by appending &defType=complexphrase I got an error given below. Any help would be appreciated thanks. java.lang.RuntimeException: java.lang.NoClassDefFoundError: org/apache/solr/search/SolrIndexReader at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:724) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:434) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.NoClassDefFoundError: org/apache/solr/search/SolrIndexReader at org.apache.solr.search.ComplexPhraseQParserPlugin.createParser(ComplexPhraseQParserPlugin.java:56) at org.apache.solr.search.QParser.getParser(QParser.java:347) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:141) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) ... 16 more Caused by: java.lang.ClassNotFoundException: org.apache.solr.search.SolrIndexReader at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at java.net.FactoryURLClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) ... 24 more
          Hide
          Ahmet Arslan added a comment -

          Hi Anonymous, It is possible that some api has changed. Tomorrow I will attach solr4.6 compatible one. In mean time you could give LUCENE-5205 a try. It has all the functionalities along with additional one. And its author Tim Allison responds very fast to any comments/questions/improvements/bugs etc. By the way SOLR-5410 is the wrapper for LUCENE-5205

          Show
          Ahmet Arslan added a comment - Hi Anonymous , It is possible that some api has changed. Tomorrow I will attach solr4.6 compatible one. In mean time you could give LUCENE-5205 a try. It has all the functionalities along with additional one. And its author Tim Allison responds very fast to any comments/questions/improvements/bugs etc. By the way SOLR-5410 is the wrapper for LUCENE-5205
          Hide
          Ahmet Arslan added a comment -

          followed the steps given in first comment.

          Anonymous, There is a ReadMe.txt in zip file. It has correct instructions.

          Show
          Ahmet Arslan added a comment - followed the steps given in first comment. Anonymous , There is a ReadMe.txt in zip file. It has correct instructions.
          Hide
          Anonymous added a comment -

          In ReadMe.txt they are putting jar file in to WEB_INF/lib dir and zipping it to new solr.war which I guess pretty much the same.

          Show
          Anonymous added a comment - In ReadMe.txt they are putting jar file in to WEB_INF/lib dir and zipping it to new solr.war which I guess pretty much the same.
          Hide
          Ahmet Arslan added a comment -

          It is mandatory because there is a package private member access and class loader problem.
          "A class loaded by one class loader can't access package-visible members in a class loaded by a different
          classloader even if they are nominally in the same package." [1]

          [1]http://www.pubbs.net/grails/200911/37981/

          Show
          Ahmet Arslan added a comment - It is mandatory because there is a package private member access and class loader problem. "A class loaded by one class loader can't access package-visible members in a class loaded by a different classloader even if they are nominally in the same package." [1] [1] http://www.pubbs.net/grails/200911/37981/
          Hide
          Anonymous added a comment -

          Now getting this error
          java.lang.NullPointerException at java.io.StringReader.<init>(Unknown Source) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:117) at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.parse(ComplexPhraseQueryParser.java:107) at org.apache.solr.search.ComplexPhraseQParser.parse(ComplexPhraseQParserPlugin.java:108) at org.apache.solr.search.QParser.getQuery(QParser.java:142) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:142) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)

          Show
          Anonymous added a comment - Now getting this error java.lang.NullPointerException at java.io.StringReader.<init>(Unknown Source) at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:117) at org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.parse(ComplexPhraseQueryParser.java:107) at org.apache.solr.search.ComplexPhraseQParser.parse(ComplexPhraseQParserPlugin.java:108) at org.apache.solr.search.QParser.getQuery(QParser.java:142) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:142) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1859) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:710) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:413) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:197) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1023) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:1852) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)
          Hide
          Anonymous added a comment -

          Issue Resolved Thanks for helping

          Show
          Anonymous added a comment - Issue Resolved Thanks for helping
          Hide
          Ahmet Arslan added a comment -

          Solr 4.7.0 compatible version.

          Show
          Ahmet Arslan added a comment - Solr 4.7.0 compatible version.
          Hide
          Erick Erickson added a comment -

          I need to look at this, so I'm assigning it to myself. That said, my knowledge of this is sparse, so I'll have a bit of a learning curve. Anyone else who wants to tackle this please feel free to take it.

          Ahmet:
          Thnkas! This couldn't have come at a better time, I was dreading dealing with this.

          Show
          Erick Erickson added a comment - I need to look at this, so I'm assigning it to myself. That said, my knowledge of this is sparse, so I'll have a bit of a learning curve. Anyone else who wants to tackle this please feel free to take it. Ahmet: Thnkas! This couldn't have come at a better time, I was dreading dealing with this.
          Hide
          Ahmet Arslan added a comment -

          Hi Erick Erickson thanks into looking into this. Let me summarise what have been so far.

          1) Consume existing ComplexPhraseQueryParser from lucene code base. Similar to lucene.SurroundQueryParser and solr.SurroundQParserPlugin relationship. This is the easiest one from solr perspective, in a way that it does not require any lucene code change. However fielded queries won't work in this solution. And you cannot set inOrder=

          {true|false}

          parameter for proximity.

          2) First fix LUCENE-1486 fielded query problem. Patch already available at LUCENE-1486. Forget about inOrder parameter.

          3) First fix both LUCENE-1486 and LUCENE-3758.

          Please let me know which is the preferred one.

          Show
          Ahmet Arslan added a comment - Hi Erick Erickson thanks into looking into this. Let me summarise what have been so far. 1) Consume existing ComplexPhraseQueryParser from lucene code base. Similar to lucene.SurroundQueryParser and solr.SurroundQParserPlugin relationship. This is the easiest one from solr perspective, in a way that it does not require any lucene code change. However fielded queries won't work in this solution. And you cannot set inOrder= {true|false} parameter for proximity. 2) First fix LUCENE-1486 fielded query problem. Patch already available at LUCENE-1486 . Forget about inOrder parameter. 3) First fix both LUCENE-1486 and LUCENE-3758 . Please let me know which is the preferred one.
          Hide
          Vitaliy Zhovtyuk added a comment -

          Probably it make sense to base plug-in on solr sources.
          Attached patch contains: org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser and org.apache.solr.search.ComplexPhraseQParserPlugin in solr and lucene sources respectively.
          Made some code cleanup.
          Added tests on lucene side for org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser.
          Renamed test resources for solr plugin test.
          I continue working on name:jo* prefix query issue when Highlighter return empty string instead of highlight matched term (TODO added).

          Show
          Vitaliy Zhovtyuk added a comment - Probably it make sense to base plug-in on solr sources. Attached patch contains: org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser and org.apache.solr.search.ComplexPhraseQParserPlugin in solr and lucene sources respectively. Made some code cleanup. Added tests on lucene side for org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser. Renamed test resources for solr plugin test. I continue working on name:jo* prefix query issue when Highlighter return empty string instead of highlight matched term (TODO added).
          Hide
          Ahmet Arslan added a comment -

          Probably it make sense to base plug-in on solr sources.

          Can you please explain what do you mean by that?

          Show
          Ahmet Arslan added a comment - Probably it make sense to base plug-in on solr sources. Can you please explain what do you mean by that?
          Hide
          Vitaliy Zhovtyuk added a comment -

          I mean 2 things:
          1. Put plug-in sources in Solr codebase (not like separate lib jar)
          2. Make org.apache.solr.search.ComplexPhraseQParserPlugin as built-in standard plug-ins

          Added patch fixing name:jo* issue with highlighter.
          Tests in org.apache.solr.search.ComplexPhraseQParserPluginTest uncommented and passing.

          Show
          Vitaliy Zhovtyuk added a comment - I mean 2 things: 1. Put plug-in sources in Solr codebase (not like separate lib jar) 2. Make org.apache.solr.search.ComplexPhraseQParserPlugin as built-in standard plug-ins Added patch fixing name:jo* issue with highlighter. Tests in org.apache.solr.search.ComplexPhraseQParserPluginTest uncommented and passing.
          Hide
          Erick Erickson added a comment -

          OK, I was looking around the patch and think I understand at least some of what's going on. To drive this forward, I need a couple of things:

          1> Vitaliy and Ahmet to resolve the two patches and let me know what the right one to use is. BTW, Vitaliy, please use "svn diff" or the equivalent Git command to create patches. Zipped up sources are much harder to work with.

          2> Some idea of a roadmap from here. Straw-man proposal:
          2a> Close 1486 and open a new JIRA if there's a fix for that if necessary. It looks to me like this patch can be committed without 1486 and we'll generate a separate fix.
          2b> commit 3758, and remove inOrder from this patch, then commit this patch.
          2c> I've assigned these to myself so I don't lose track of them. I'll look desperately for cycles to work on them . But I have a couple of long plane flights in my future...

          3> Of course we need to document the syntax and behavior here, Cassandra Targett can probably point us in the right direction for doing this right by putting it in the new documentation!

          4> I'm also curious what we know now in terms of performance, resource requirements, that kind of stuff.

          5> I notice there's a patch labeled as having to do with license stuff. What's up there? Is this just putting the headers in the source files?

          5> Anything else? Does anyone out there object to moving forward with this?

          Show
          Erick Erickson added a comment - OK, I was looking around the patch and think I understand at least some of what's going on. To drive this forward, I need a couple of things: 1> Vitaliy and Ahmet to resolve the two patches and let me know what the right one to use is. BTW, Vitaliy, please use "svn diff" or the equivalent Git command to create patches. Zipped up sources are much harder to work with. 2> Some idea of a roadmap from here. Straw-man proposal: 2a> Close 1486 and open a new JIRA if there's a fix for that if necessary. It looks to me like this patch can be committed without 1486 and we'll generate a separate fix. 2b> commit 3758, and remove inOrder from this patch, then commit this patch. 2c> I've assigned these to myself so I don't lose track of them. I'll look desperately for cycles to work on them . But I have a couple of long plane flights in my future... 3> Of course we need to document the syntax and behavior here, Cassandra Targett can probably point us in the right direction for doing this right by putting it in the new documentation! 4> I'm also curious what we know now in terms of performance, resource requirements, that kind of stuff. 5> I notice there's a patch labeled as having to do with license stuff. What's up there? Is this just putting the headers in the source files? 5> Anything else? Does anyone out there object to moving forward with this?
          Hide
          Ahmet Arslan added a comment -

          Here are some clarification regarding zipped attachments :

          Zipped attachments are not meant for source code inclusion but for to be consumed as solr plugin. They will never be committed. Mainly because zipped version(s) include a duplicate code from lucene code base. Duplicated class is org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser. Duplication is done for two reasons :

          • To enable fielded queries. this duplicate code changes package name to org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser. Originally Somehow this feature forgotten accidentally in LUCENE1468, while committing lucene.ComplexPhraseQueryParser. After that commit, package name changed from classic to complexPhrase. For this fix it needs to access a field from super class. After realizing this, chancing this fields visibility to protected is accepted by lazy consensus. This is the patch for this.
          • To enable ability change inOrder parameter. In original lucene code inOrder parameter is barcoded to true inSpanNearQuery classes. Separate jira for this is LUCENE-3758.

          By the way, why LUCENE-1486 is re-opened is a mystery. It is not re-opened because of unforgotten non-default patch.

          Show
          Ahmet Arslan added a comment - Here are some clarification regarding zipped attachments : Zipped attachments are not meant for source code inclusion but for to be consumed as solr plugin. They will never be committed. Mainly because zipped version(s) include a duplicate code from lucene code base. Duplicated class is org.apache.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser. Duplication is done for two reasons : To enable fielded queries. this duplicate code changes package name to org.apache.lucene.queryparser.classic.ComplexPhraseQueryParser. Originally Somehow this feature forgotten accidentally in LUCENE1468, while committing lucene.ComplexPhraseQueryParser. After that commit, package name changed from classic to complexPhrase. For this fix it needs to access a field from super class. After realizing this, chancing this fields visibility to protected is accepted by lazy consensus. This is the patch for this. To enable ability change inOrder parameter. In original lucene code inOrder parameter is barcoded to true inSpanNearQuery classes. Separate jira for this is LUCENE-3758 . By the way, why LUCENE-1486 is re-opened is a mystery. It is not re-opened because of unforgotten non-default patch.
          Hide
          Ahmet Arslan added a comment -

          Vitaliy and Ahmet to resolve the two patches and let me know what the right one to use is.

          none of them actually. They include source code (ComplexPhraseQueryParser.java) duplication from lucene. I will attach a patch that consumes lucene's ComplexPhraseQueryParser created against trunk.

          Close 1486 and open a new JIRA if there's a fix for that if necessary. It looks to me like this patch can be committed without 1486 and we'll generate a separate fix.

          +1. Yes this patch can be committed without LUCENE-1486. +1 for closing LUCENE-1486 given that it is re-opened mysteriously. +1 for creating a separate jira for this functionality just because it is less confusing.

          commit 3758, and remove inOrder from this patch, then commit this patch.

          Request ability change inOrder parameter came from a user. Robert had this comment about this.

          I notice there's a patch labeled as having to do with license stuff.

          This attachment is old. I accidentally forget selection 'ASF inclusion" radio box then. Jira weren't displaying feather icon for that. After that incident jira had removed that radio button selection option. Attachments are ASF granted by default now. That file is renamed automatically by infra.

          Show
          Ahmet Arslan added a comment - Vitaliy and Ahmet to resolve the two patches and let me know what the right one to use is. none of them actually. They include source code (ComplexPhraseQueryParser.java) duplication from lucene. I will attach a patch that consumes lucene's ComplexPhraseQueryParser created against trunk. Close 1486 and open a new JIRA if there's a fix for that if necessary. It looks to me like this patch can be committed without 1486 and we'll generate a separate fix. +1. Yes this patch can be committed without LUCENE-1486 . +1 for closing LUCENE-1486 given that it is re-opened mysteriously. +1 for creating a separate jira for this functionality just because it is less confusing. commit 3758, and remove inOrder from this patch, then commit this patch. Request ability change inOrder parameter came from a user. Robert had this comment about this. I notice there's a patch labeled as having to do with license stuff. This attachment is old. I accidentally forget selection 'ASF inclusion" radio box then. Jira weren't displaying feather icon for that. After that incident jira had removed that radio button selection option. Attachments are ASF granted by default now. That file is renamed automatically by infra.
          Hide
          Ahmet Arslan added a comment -

          This is solr-only patch (solr/core/src/) and does not touch lucene code case. It adds two new java classes (ComplexPhraseQParserPlugin and TestComplexPhraseQParserPlugin) and consumes o.a.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser

          Show
          Ahmet Arslan added a comment - This is solr-only patch (solr/core/src/) and does not touch lucene code case. It adds two new java classes (ComplexPhraseQParserPlugin and TestComplexPhraseQParserPlugin) and consumes o.a.lucene.queryparser.complexPhrase.ComplexPhraseQueryParser
          Hide
          Erick Erickson added a comment -

          Added entry to CHANGES.txt for Solr

          Show
          Erick Erickson added a comment - Added entry to CHANGES.txt for Solr
          Hide
          Ahmet Arslan added a comment -

          QueryEqualityTest added.

          Show
          Ahmet Arslan added a comment - QueryEqualityTest added.
          Hide
          Ahmet Arslan added a comment -

          following commands pass for me with last patch.

          • ant -Dtests.disableHdfs=true -Dtests.badapples=false test
          • ant -Dtestcase=QueryEqualityTest test
          • ant -Dtestcase=TestComplexPhraseQuery test
          • ant -Dtestcase=TestComplexPhraseQParserPlugin test
          Show
          Ahmet Arslan added a comment - following commands pass for me with last patch. ant -Dtests.disableHdfs=true -Dtests.badapples=false test ant -Dtestcase=QueryEqualityTest test ant -Dtestcase=TestComplexPhraseQuery test ant -Dtestcase=TestComplexPhraseQParserPlugin test
          Hide
          ASF subversion and git services added a comment -

          Commit 1578200 from Erick Erickson in branch 'dev/trunk'
          [ https://svn.apache.org/r1578200 ]

          SOLR-1604: Wildcards, ORs etc inside Phrase Queries or 'ComplexPhraseQueryParser support in Solr'

          Show
          ASF subversion and git services added a comment - Commit 1578200 from Erick Erickson in branch 'dev/trunk' [ https://svn.apache.org/r1578200 ] SOLR-1604 : Wildcards, ORs etc inside Phrase Queries or 'ComplexPhraseQueryParser support in Solr'
          Hide
          ASF subversion and git services added a comment -

          Commit 1578218 from Erick Erickson in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1578218 ]

          SOLR-1604: Wildcards, ORs etc inside Phrase Queries or 'ComplexPhraseQueryParser support in Solr'

          Show
          ASF subversion and git services added a comment - Commit 1578218 from Erick Erickson in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1578218 ] SOLR-1604 : Wildcards, ORs etc inside Phrase Queries or 'ComplexPhraseQueryParser support in Solr'
          Hide
          Erick Erickson added a comment -

          OK, quite a bit has been checked in for this, but I think it's all done. Let's see what happens now!

          Show
          Erick Erickson added a comment - OK, quite a bit has been checked in for this, but I think it's all done. Let's see what happens now!
          Hide
          Erick Erickson added a comment -

          OK, to finish this off, we need a Wiki/Confluence page, calling for volunteers:

          Some points that should be mentioned I think:
          > how to set up/use (simple really, defType)
          > A number of examples
          > inOrder="true|false" as a local param mentioned
          > Anyone's experience with how it performs, especially with things like single-letter wildcards (e.g. "j* smith")

          Show
          Erick Erickson added a comment - OK, to finish this off, we need a Wiki/Confluence page, calling for volunteers: Some points that should be mentioned I think: > how to set up/use (simple really, defType) > A number of examples > inOrder="true|false" as a local param mentioned > Anyone's experience with how it performs, especially with things like single-letter wildcards (e.g. "j* smith")
          Hide
          Ahmet Arslan added a comment -

          Thanks Erick Erickson for committing this! Here is what I compiled from README.txt from zip distro.

          After indexing example documents under example/exampledocs via 'java -jar post.jar *.xml' utility

          The query string

          q=manu:"a* c*"&defType=complexphrase

          or

          q={!complexphrase inOrder=true}manu:"a* c*" 

          will return :

          http://localhost:8983/solr/collection1/select?q=manu:%22a*%20c*%22&defType=complexphrase&fl=manu

          <doc>
            <str name="manu">Apple Computer Inc.</str>
          </doc>
          <doc>
            <str name="manu">ASUS Computer Inc.</str>
          </doc>
          

          inOrder Parameter can be set in two ways.

          1) Its default value is true. If you want to set it to false in a permanent way : register query parser with a different name in solrconfig.xml

           <!-- Un-ordered complex phrase query parser -->
            <queryParser name="unorderedcomplexphrase" class="org.apache.solr.search.ComplexPhraseQParserPlugin">
              <bool name="inOrder">false</bool>
            </queryParser>
          

          2) At query time via local params.

          q={!complexphrase inOrder=false df=name}"bla* pla*"

          To mix ordered and unordered clauses in the same query.

          +_query_:"{!complexphrase inOrder=true}manu:\"a* c*\""  +_query_:"{!complexphrase inOrder=false df=name}\"bla* pla*\""  
          

          Limitations

          maxBooleanClauses

          You may need to increase

          <maxBooleanClauses>1024</maxBooleanClauses>

          according to index size in solrconfig.xml because

          "a* c*"

          is expanded into SpanNearQuery

          spanNear([spanOr([manu:a, manu:america, manu:apache, manu:apple, manu:asus, manu:ati]), spanOr([manu:canon, manu:co, manu:computer, manu:corp, manu:corsair])], 0, false)
          
          Stopwords

          Lets say we add the, up, to to collection1/conf/stopwords.txt file and re-index example docs.
          While

          q=features:"Stores up to 15,000"

          returns "Stores up to 15,000 songs, 25,000 photos, or 150 hours of video",

          q=features:"sto* up to 15*"&defType=complexphrase 

          does not return that document because SpanNearQuery has no good way to handle stopwords in a way analagous to PhraseQuery. It is recommended not to use stopword elimination with this query parser.

          Show
          Ahmet Arslan added a comment - Thanks Erick Erickson for committing this! Here is what I compiled from README.txt from zip distro. After indexing example documents under example/exampledocs via 'java -jar post.jar *.xml' utility The query string q=manu:"a* c*"&defType=complexphrase or q={!complexphrase inOrder=true}manu:"a* c*" will return : http://localhost:8983/solr/collection1/select?q=manu:%22a*%20c*%22&defType=complexphrase&fl=manu <doc> <str name= "manu" > Apple Computer Inc. </str> </doc> <doc> <str name= "manu" > ASUS Computer Inc. </str> </doc> inOrder Parameter can be set in two ways. 1) Its default value is true. If you want to set it to false in a permanent way : register query parser with a different name in solrconfig.xml <!-- Un-ordered complex phrase query parser --> <queryParser name= "unorderedcomplexphrase" class= "org.apache.solr.search.ComplexPhraseQParserPlugin" > <bool name= "inOrder" > false </bool> </queryParser> 2) At query time via local params. q={!complexphrase inOrder=false df=name}"bla* pla*" To mix ordered and unordered clauses in the same query. +_query_:"{!complexphrase inOrder=true}manu:\"a* c*\"" +_query_:"{!complexphrase inOrder=false df=name}\"bla* pla*\"" Limitations maxBooleanClauses You may need to increase <maxBooleanClauses> 1024 </maxBooleanClauses> according to index size in solrconfig.xml because "a* c*" is expanded into SpanNearQuery spanNear([spanOr([manu:a, manu:america, manu:apache, manu:apple, manu:asus, manu:ati]), spanOr([manu:canon, manu:co, manu:computer, manu:corp, manu:corsair])], 0, false) Stopwords Lets say we add the , up , to to collection1/conf/stopwords.txt file and re-index example docs. While q=features:"Stores up to 15,000" returns "Stores up to 15,000 songs, 25,000 photos, or 150 hours of video" , q=features:"sto* up to 15*"&defType=complexphrase does not return that document because SpanNearQuery has no good way to handle stopwords in a way analagous to PhraseQuery. It is recommended not to use stopword elimination with this query parser.
          Hide
          Anonymous added a comment -

          Hey,
          I am using ComplexPhrase 4.2.1 with solr 4.7.0
          Is there any difference between ComplexPhrase 4.7.0 and ComplexPhrase 4.2.1?
          And also
          Can I run this type of query "hello world h*" with this parser?

          Show
          Anonymous added a comment - Hey, I am using ComplexPhrase 4.2.1 with solr 4.7.0 Is there any difference between ComplexPhrase 4.7.0 and ComplexPhrase 4.2.1? And also Can I run this type of query "hello world h*" with this parser?
          Hide
          Ahmet Arslan added a comment -

          Hi Anonymous, there is no difference between 4.7.0 and 4.2.1. Committed one has local param addition. You can set inOrder parameter using http://wiki.apache.org/solr/LocalParams

          And yes "hello world h*" is perfectly fine.

          Show
          Ahmet Arslan added a comment - Hi Anonymous , there is no difference between 4.7.0 and 4.2.1. Committed one has local param addition. You can set inOrder parameter using http://wiki.apache.org/solr/LocalParams And yes "hello world h*" is perfectly fine.
          Hide
          Anonymous added a comment -

          Thanks Ahmet,
          Yeah I saw that using debugQuery On and found that even phrase query is executed using "Spannear query" the addition of wildcard makes it to use "spanOR" inside "spannear query".
          One more Question:
          when spannear is given multiple words like spannear(hello,world,hello) with slop "0" how it is working inside to find all the documents containing all above terms adjacent to eachother?

          Show
          Anonymous added a comment - Thanks Ahmet, Yeah I saw that using debugQuery On and found that even phrase query is executed using "Spannear query" the addition of wildcard makes it to use "spanOR" inside "spannear query". One more Question: when spannear is given multiple words like spannear(hello,world,hello) with slop "0" how it is working inside to find all the documents containing all above terms adjacent to eachother?
          Hide
          Ahmet Arslan added a comment -

          when spannear is given multiple words like spannear(hello,world,hello) with slop "0"

          With inOrder=true or inOrder=false?

          I am sure what happens, when you use the same term twice. I remember someone asked similar question on lucene list. http://search-lucene.com/m/SnDCWotpUB1 Span*Query family less used so there can be gotchas. Can you please tell us this treat is related your use case?

          Show
          Ahmet Arslan added a comment - when spannear is given multiple words like spannear(hello,world,hello) with slop "0" With inOrder=true or inOrder=false? I am sure what happens, when you use the same term twice. I remember someone asked similar question on lucene list. http://search-lucene.com/m/SnDCWotpUB1 Span*Query family less used so there can be gotchas. Can you please tell us this treat is related your use case?
          Hide
          Anonymous added a comment -

          inorder=true
          I am asking for general use case there can be different terms also.

          Show
          Anonymous added a comment - inorder=true I am asking for general use case there can be different terms also.
          Hide
          Erick Erickson added a comment -

          Let's move this discussion to the user's list. We're in to "how to use this" discussion I think. That way we'll get a wider audience for this.

          Show
          Erick Erickson added a comment - Let's move this discussion to the user's list. We're in to "how to use this" discussion I think. That way we'll get a wider audience for this.
          Hide
          Ahmet Arslan added a comment -

          Hi Erick Erickson, yes you are right about user's mailing list.
          By the way did you have a chance look into documentation candidate I wrote?

          Show
          Ahmet Arslan added a comment - Hi Erick Erickson , yes you are right about user's mailing list. By the way did you have a chance look into documentation candidate I wrote?
          Hide
          Erick Erickson added a comment -

          Ahmet Arslan About the doc candidate. I took a quick glance at it, and I say go for it.

          This looks like a good intro. Any shortcomings will become immediately
          apparent and we'll fix it. The only documentation I really object to is false
          documentation .

          So feel free to put it up on the Wiki. If I could ask you to, after you do create a JIRA
          and assign it to me to put the data over on the CWiki...

          Thanks!

          Show
          Erick Erickson added a comment - Ahmet Arslan About the doc candidate. I took a quick glance at it, and I say go for it. This looks like a good intro. Any shortcomings will become immediately apparent and we'll fix it. The only documentation I really object to is false documentation . So feel free to put it up on the Wiki. If I could ask you to, after you do create a JIRA and assign it to me to put the data over on the CWiki... Thanks!
          Hide
          Ahmet Arslan added a comment -

          feel free to put it up on the Wiki.

          Done. http://wiki.apache.org/solr/ComplexPhraseQueryParser

          create a JIRA and assign it to me to put the data over on the CWiki…

          Done. SOLR-5926

          Show
          Ahmet Arslan added a comment - feel free to put it up on the Wiki. Done. http://wiki.apache.org/solr/ComplexPhraseQueryParser create a JIRA and assign it to me to put the data over on the CWiki… Done. SOLR-5926
          Hide
          Uwe Schindler added a comment -

          Close issue after release of 4.8.0

          Show
          Uwe Schindler added a comment - Close issue after release of 4.8.0

            People

            • Assignee:
              Erick Erickson
              Reporter:
              Ahmet Arslan
            • Votes:
              19 Vote for this issue
              Watchers:
              34 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development