Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10357

When sow=false, edismax&standard query parsers should handle per-fieldtype autoGeneratePhraseQueries by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.6, 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Right now, the options to not split on whitespace (sow=false) and to autogenerate phrase queries (autoGeneratePhraseQueries="true") will cause queries to throw an exception, since they are incompatible.

      QueryBuilder.autoGenerateMultiTermSynonymsPhraseQuery, introduced in LUCENE-7638, is the graph query version of Solr's per-fieldtype autoGeneratePhraseQueries option, and is not incompatible with sow=false.

      So autoGeneratePhraseQueries="true" should cause QueryBuilder.autoGenerateMultiTermSynonymsPhraseQuery to be set to true when sow=false, rather than triggering an exception.

      1. SOLR-10357.patch
        25 kB
        Steve Rowe
      2. SOLR-10357.patch
        25 kB
        Steve Rowe
      3. SOLR-10357.patch
        26 kB
        Steve Rowe

        Issue Links

          Activity

          Hide
          dsmiley David Smiley added a comment -

          +1 to the idea.

          Show
          dsmiley David Smiley added a comment - +1 to the idea.
          Hide
          steve_rowe Steve Rowe added a comment - - edited

          Patch implementing the idea.

          There remains an issue: the original auto-gen use case (for e.g. wifi => wi fi, replacing a single token with multiple tokens, but no overlapping terms) still doesn't auto-generate a phrase query for wi fi if synonyms are replaced rather than expanded when QueryBuilder.autoGenerateMultiTermSynonymsQuery is set to true. So this is not (yet) a complete solution. IMHO this is a weakness of how QueryBuilder decides whether to invoke "graph" processing, currently only when a posLength > 1 is encountered (as when there are overlapping multi-word synonyms).

          This patch changes the test schema named schema12.xml to use the graph-aware versions of the synonym and word delimiter filters. Instead of committing as-is (pending full testing and precommit, not yet done), I'll do the rote schema conversions on SOLR-10343 and SOLR-10344 and set those as blockers of this issue; that seems like a more appropriate way to handle whatever conflicts might come up from other uses of schema12.xml.

          Show
          steve_rowe Steve Rowe added a comment - - edited Patch implementing the idea. There remains an issue: the original auto-gen use case (for e.g. wifi => wi fi , replacing a single token with multiple tokens, but no overlapping terms) still doesn't auto-generate a phrase query for wi fi if synonyms are replaced rather than expanded when QueryBuilder.autoGenerateMultiTermSynonymsQuery is set to true . So this is not (yet) a complete solution. IMHO this is a weakness of how QueryBuilder decides whether to invoke "graph" processing, currently only when a posLength > 1 is encountered (as when there are overlapping multi-word synonyms). This patch changes the test schema named schema12.xml to use the graph-aware versions of the synonym and word delimiter filters. Instead of committing as-is (pending full testing and precommit, not yet done), I'll do the rote schema conversions on SOLR-10343 and SOLR-10344 and set those as blockers of this issue; that seems like a more appropriate way to handle whatever conflicts might come up from other uses of schema12.xml .
          Hide
          steve_rowe Steve Rowe added a comment -

          Updated patch after committing SOLR-10343 and SOLR-10344.

          Running tests and precommit now.

          Show
          steve_rowe Steve Rowe added a comment - Updated patch after committing SOLR-10343 and SOLR-10344 . Running tests and precommit now.
          Hide
          steve_rowe Steve Rowe added a comment -

          Final patch. Previous patches were missing QueryParser.java changes (I directly changed this file, then regenerated from QueryParser.jj, then made a patch...)

          Solr tests and precommit pass. Committing shortly.

          Show
          steve_rowe Steve Rowe added a comment - Final patch. Previous patches were missing QueryParser.java changes (I directly changed this file, then regenerated from QueryParser.jj, then made a patch...) Solr tests and precommit pass. Committing shortly.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit da2cfda02fe539c42f1794fc56a478a3acc7d111 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=da2cfda ]

          SOLR-10357: Enable edismax and standard query parsers to handle the option combination sow=false / autoGeneratePhraseQueries=true by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery

          Show
          jira-bot ASF subversion and git services added a comment - Commit da2cfda02fe539c42f1794fc56a478a3acc7d111 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=da2cfda ] SOLR-10357 : Enable edismax and standard query parsers to handle the option combination sow=false / autoGeneratePhraseQueries=true by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 0a689f4d95e981e99ae0e80741e7cf1fa74ff60f in lucene-solr's branch refs/heads/master from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0a689f4 ]

          SOLR-10357: Enable edismax and standard query parsers to handle the option combination sow=false / autoGeneratePhraseQueries=true by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery

          Show
          jira-bot ASF subversion and git services added a comment - Commit 0a689f4d95e981e99ae0e80741e7cf1fa74ff60f in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0a689f4 ] SOLR-10357 : Enable edismax and standard query parsers to handle the option combination sow=false / autoGeneratePhraseQueries=true by setting QueryBuilder.autoGenerateMultiTermSynonymsQuery
          Hide
          dsmiley David Smiley added a comment -

          Thanks Steve.

          Show
          dsmiley David Smiley added a comment - Thanks Steve.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 7d164a1dbe825703d835aa912edd98b660e2a8c4 in lucene-solr's branch refs/heads/branch_6_6 from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7d164a1 ]

          SOLR-10357: add attribution to CHANGES entry

          Show
          jira-bot ASF subversion and git services added a comment - Commit 7d164a1dbe825703d835aa912edd98b660e2a8c4 in lucene-solr's branch refs/heads/branch_6_6 from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7d164a1 ] SOLR-10357 : add attribution to CHANGES entry
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f0e6e89a420b8f8f98aecc69a4d459a409f0d09c in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f0e6e89 ]

          SOLR-10357: add attribution to CHANGES entry

          Show
          jira-bot ASF subversion and git services added a comment - Commit f0e6e89a420b8f8f98aecc69a4d459a409f0d09c in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f0e6e89 ] SOLR-10357 : add attribution to CHANGES entry
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 4cc20d27cd2da40801ca15ba1478a43a1a121781 in lucene-solr's branch refs/heads/master from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4cc20d2 ]

          SOLR-10357: add attribution to CHANGES entry

          Show
          jira-bot ASF subversion and git services added a comment - Commit 4cc20d27cd2da40801ca15ba1478a43a1a121781 in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4cc20d2 ] SOLR-10357 : add attribution to CHANGES entry

            People

            • Assignee:
              steve_rowe Steve Rowe
              Reporter:
              steve_rowe Steve Rowe
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development