Solr
  1. Solr
  2. SOLR-3960

Searching: PostFilter plugins ignored by solr cloud for queries with grouping

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0-BETA
    • Fix Version/s: 4.1, 6.0
    • Component/s: search, SolrCloud
    • Labels:
    • Environment:

      Linux / jre 1.6

      Description

      When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup parameters, solr search does not consider any PostFilters.
      For example, I have implemented a PostFilter and wrapped that in my QParserPlugin sub class. Then in the solr query I am using fq to apply post filtering by specifying fq=

      {!pf1}

      where pf1 is the name of my QParserPlugin. This all works well in solr and not in solr cloud.
      I debugged and figured out that org.apache.solr.search.Grouping.execute() method is called by the non cloud instance. This considers pf.PostFilter which is the collection of all PostFilters whereas org.apache.solr.search.grouping.CommandHandler.execute() which is called by the solr cloud instance ignores the pf.PostFilter.

      1. SOLR-3960.patch
        6 kB
        Hoss Man
      2. SOLR-3960.patch
        6 kB
        Hoss Man

        Activity

        Hide
        Hoss Man added a comment -

        Nathan: just to clarify, you said...

        When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup parameters, solr search does not consider any PostFilters.

        but then you go on to say...

        I debugged and figured out that org.apache.solr.search.Grouping.execute() method is called by the non cloud instance. This considers pf.PostFilter which is the collection of all PostFilters whereas org.apache.solr.search.grouping.CommandHandler.execute()

        which makes it sound like this problem is specific to using the grouping features combined with distributed search (ie: solr cloud) and not a general problem with all distributed search.

        can you please confirm/clarify that? when you do a regular search, w/o using grouping, then do you see your postfilter used in solrcloud?

        It would also help if you could provide a simple example of the problem you are seeing that didn't require using your custom component QParserPlugin – so we can rule out that there isn't anything odd in how your custom code. a JUnit test would be ideal, but even providing an example URL using the example configs that are included with solr showing the problem would be helpful. (eg: the frange qparser can be used as a PostFilter)

        Show
        Hoss Man added a comment - Nathan: just to clarify, you said... When specifying the cloud options (-DzkRun -DnumShards=n) in solr startup parameters, solr search does not consider any PostFilters. but then you go on to say... I debugged and figured out that org.apache.solr.search.Grouping.execute() method is called by the non cloud instance. This considers pf.PostFilter which is the collection of all PostFilters whereas org.apache.solr.search.grouping.CommandHandler.execute() which makes it sound like this problem is specific to using the grouping features combined with distributed search (ie: solr cloud) and not a general problem with all distributed search. can you please confirm/clarify that? when you do a regular search, w/o using grouping, then do you see your postfilter used in solrcloud? It would also help if you could provide a simple example of the problem you are seeing that didn't require using your custom component QParserPlugin – so we can rule out that there isn't anything odd in how your custom code. a JUnit test would be ideal, but even providing an example URL using the example configs that are included with solr showing the problem would be helpful. (eg: the frange qparser can be used as a PostFilter)
        Hide
        Nathan Visagan added a comment - - edited

        Hoss, You are correct. Its only happening with grouping and solr cloud options. For example if I send a query like q=:&group=true&group.field=language&fq=

        {!pf1}

        .
        Sorry. The confusion earlier happened because I had some default grouping enabled for all the queries in the solrconfig select handler.
        Updated the title accordingly.

        Show
        Nathan Visagan added a comment - - edited Hoss, You are correct. Its only happening with grouping and solr cloud options. For example if I send a query like q= : &group=true&group.field=language&fq= {!pf1} . Sorry. The confusion earlier happened because I had some default grouping enabled for all the queries in the solrconfig select handler. Updated the title accordingly.
        Hide
        Hoss Man added a comment -

        grouping and postfilters are two areas of internals that i'm not really familiar with, but based on Nathan's problem description and great job pointing out the methods he noticed not playing nicely, i was able to create a test case demonstrating the problem, and then take a blind stab at a fix.

        would definitely appreciate more eyeballs on this

        Show
        Hoss Man added a comment - grouping and postfilters are two areas of internals that i'm not really familiar with, but based on Nathan's problem description and great job pointing out the methods he noticed not playing nicely, i was able to create a test case demonstrating the problem, and then take a blind stab at a fix. would definitely appreciate more eyeballs on this
        Hide
        Hoss Man added a comment -

        slight test improvement to exercise distributed grouping both with a regular filter and with a post-filter

        Show
        Hoss Man added a comment - slight test improvement to exercise distributed grouping both with a regular filter and with a post-filter
        Hide
        Yonik Seeley added a comment -

        Looks good Hoss!

        Show
        Yonik Seeley added a comment - Looks good Hoss!
        Hide
        Commit Tag Bot added a comment -

        [trunk commit] Chris M. Hostetter
        http://svn.apache.org/viewvc?view=revision&revision=1413975

        SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters

        Show
        Commit Tag Bot added a comment - [trunk commit] Chris M. Hostetter http://svn.apache.org/viewvc?view=revision&revision=1413975 SOLR-3960 : Fixed a bug where Distributed Grouping ignored PostFilters
        Hide
        Hoss Man added a comment -

        Thanks for hte review yonik, and thanks Nathan for the bug report.

        Committed revision 1413975. - trunk
        Committed revision 1413983. - 4x

        Show
        Hoss Man added a comment - Thanks for hte review yonik, and thanks Nathan for the bug report. Committed revision 1413975. - trunk Committed revision 1413983. - 4x
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] Chris M. Hostetter
        http://svn.apache.org/viewvc?view=revision&revision=1413983

        SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters (merge r1413975)

        Show
        Commit Tag Bot added a comment - [branch_4x commit] Chris M. Hostetter http://svn.apache.org/viewvc?view=revision&revision=1413983 SOLR-3960 : Fixed a bug where Distributed Grouping ignored PostFilters (merge r1413975)
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] Chris M. Hostetter
        http://svn.apache.org/viewvc?view=revision&revision=1413983

        SOLR-3960: Fixed a bug where Distributed Grouping ignored PostFilters (merge r1413975)

        Show
        Commit Tag Bot added a comment - [branch_4x commit] Chris M. Hostetter http://svn.apache.org/viewvc?view=revision&revision=1413983 SOLR-3960 : Fixed a bug where Distributed Grouping ignored PostFilters (merge r1413975)

          People

          • Assignee:
            Hoss Man
            Reporter:
            Nathan Visagan
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development