Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.3
    • Fix Version/s: 3.4, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None

      Description

      Once Lucene is wired, add this feature to Solr.

      1. SOLR-2665.patch
        15 kB
        Martijn van Groningen
      2. SOLR-2665.patch
        16 kB
        Tomás Fernández Löbbe
      3. SOLR-2665.patch
        16 kB
        Martijn van Groningen
      4. SOLR-2665.patch
        21 kB
        Martijn van Groningen

        Issue Links

          Activity

          Gavin made changes -
          Link This issue depends upon LUCENE-3097 [ LUCENE-3097 ]
          Gavin made changes -
          Link This issue depends on LUCENE-3097 [ LUCENE-3097 ]
          Hide
          Martijn van Groningen added a comment -

          Martijn. Can you make a new issue with the code you already had for this maybe?

          I created the following issue for this: SOLR-2898

          Show
          Martijn van Groningen added a comment - Martijn. Can you make a new issue with the code you already had for this maybe? I created the following issue for this: SOLR-2898
          Hide
          Okke Klein added a comment -

          With LUCENE-3097 resolved, can we get real post grouping faceting(matrix counts)?

          Martijn. Can you make a new issue with the code you already had for this maybe?

          Show
          Okke Klein added a comment - With LUCENE-3097 resolved, can we get real post grouping faceting(matrix counts)? Martijn. Can you make a new issue with the code you already had for this maybe?
          Hide
          Martijn van Groningen added a comment -

          The parameter to use is:
          group.truncate

          See wiki page for more info:
          http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters

          Show
          Martijn van Groningen added a comment - The parameter to use is: group.truncate See wiki page for more info: http://wiki.apache.org/solr/FieldCollapsing#Request_Parameters
          Hide
          Shaul Zevin added a comment -

          Hi Guys,

          What is the parameter to be used? Is it group.after ?
          Can somebody add an example query.

          Thanks,
          Shaul

          Show
          Shaul Zevin added a comment - Hi Guys, What is the parameter to be used? Is it group.after ? Can somebody add an example query. Thanks, Shaul
          Robert Muir made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Robert Muir added a comment -

          bulk close for 3.4

          Show
          Robert Muir added a comment - bulk close for 3.4
          Martijn van Groningen made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Martijn van Groningen added a comment -

          Committed.
          Trunk: r1154676
          Branch3x: r1154675

          Show
          Martijn van Groningen added a comment - Committed. Trunk: r1154676 Branch3x: r1154675
          Martijn van Groningen made changes -
          Attachment SOLR-2665.patch [ 12489185 ]
          Hide
          Martijn van Groningen added a comment -

          Attached updated patch.

          I hooked group.truncate into random grouping test. This revealed some bugs which have been fixed. I think we're getting close to get this committed! I think in the coming days I'll commit this feature to trunk and 3x.

          Show
          Martijn van Groningen added a comment - Attached updated patch. I hooked group.truncate into random grouping test. This revealed some bugs which have been fixed. I think we're getting close to get this committed! I think in the coming days I'll commit this feature to trunk and 3x.
          Martijn van Groningen made changes -
          Attachment SOLR-2665.patch [ 12488130 ]
          Hide
          Martijn van Groningen added a comment -

          Updated the patch.

          • The parameter name has changed from group.after to group.truncate.
          • Also removed the changes in testGroupingSortByFunction test method.
          Show
          Martijn van Groningen added a comment - Updated the patch. The parameter name has changed from group.after to group.truncate. Also removed the changes in testGroupingSortByFunction test method.
          Hide
          Martijn van Groningen added a comment - - edited

          I'm uploading a patch isolating this error and a fix.

          Thanks Tomas!

          Also, I don't know why, but the test case testGroupingSortByFunction with the extra line you added on the previous patch
          assertU(add(doc("id", "6", "value1_i", "1", "value2_i", "5", "store", "45.18014,-93.87746")));
          didn't run correctly for me, I'm not sure why should it run. I didn't touch that line but that test doesn't run for me.

          Oops. That is a copy / past error. That test shouldn't never have been changed.

          If this essentially drops documents after the group.limit in each group, how about naming the parameter group.truncate?

          I think that is a better name, than there is now. I'll change the group.after parameter into group.trancate.

          Show
          Martijn van Groningen added a comment - - edited I'm uploading a patch isolating this error and a fix. Thanks Tomas! Also, I don't know why, but the test case testGroupingSortByFunction with the extra line you added on the previous patch assertU(add(doc("id", "6", "value1_i", "1", "value2_i", "5", "store", "45.18014,-93.87746"))); didn't run correctly for me, I'm not sure why should it run. I didn't touch that line but that test doesn't run for me. Oops. That is a copy / past error. That test shouldn't never have been changed. If this essentially drops documents after the group.limit in each group, how about naming the parameter group.truncate? I think that is a better name, than there is now. I'll change the group.after parameter into group.trancate.
          Hide
          Yonik Seeley added a comment -

          If this essentially drops documents after the group.limit in each group, how about naming the parameter group.truncate?

          Show
          Yonik Seeley added a comment - If this essentially drops documents after the group.limit in each group, how about naming the parameter group.truncate?
          Tomás Fernández Löbbe made changes -
          Attachment SOLR-2665.patch [ 12488128 ]
          Hide
          Tomás Fernández Löbbe added a comment -

          Martijn, current patch breaks when the group field is not a String and there is no specific sort. This is the stacktrace of the Exception:

          java.lang.NullPointerException
          at org.apache.solr.search.Grouping$FunctionAllGroupHeadsCollector.<init>(Grouping.java:1156)
          at org.apache.solr.search.Grouping$CommandFunc.createAllGroupCollector(Grouping.java:920)
          at org.apache.solr.search.Grouping.execute(Grouping.java:317)
          at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:376)
          at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194)
          at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
          at org.apache.solr.core.SolrCore.execute(SolrCore.java:1401)
          at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248)
          at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
          at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
          at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
          at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
          at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
          at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
          at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
          at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
          at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
          at org.mortbay.jetty.Server.handle(Server.java:326)
          at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
          at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
          at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
          at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
          at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
          at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
          at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)

          I got to this exception by using Solr's example schema/data and this request:
          http://localhost:8983/solr/select?q=*:*&group=true&group.field=popularity&facet.field=inStock&facet=true&group.after=true

          I'm uploading a patch isolating this error and a fix.

          Also, I don't know why, but the test case testGroupingSortByFunction with the extra line you added on the previous patch
          assertU(add(doc("id", "6", "value1_i", "1", "value2_i", "5", "store", "45.18014,-93.87746")));
          didn't run correctly for me, I'm not sure why should it run. I didn't touch that line but that test doesn't run for me.

          Show
          Tomás Fernández Löbbe added a comment - Martijn, current patch breaks when the group field is not a String and there is no specific sort. This is the stacktrace of the Exception: java.lang.NullPointerException at org.apache.solr.search.Grouping$FunctionAllGroupHeadsCollector.<init>(Grouping.java:1156) at org.apache.solr.search.Grouping$CommandFunc.createAllGroupCollector(Grouping.java:920) at org.apache.solr.search.Grouping.execute(Grouping.java:317) at org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:376) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:194) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1401) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:353) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:248) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450) at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230) at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) I got to this exception by using Solr's example schema/data and this request: http://localhost:8983/solr/select?q=*:*&group=true&group.field=popularity&facet.field=inStock&facet=true&group.after=true I'm uploading a patch isolating this error and a fix. Also, I don't know why, but the test case testGroupingSortByFunction with the extra line you added on the previous patch assertU(add(doc("id", "6", "value1_i", "1", "value2_i", "5", "store", "45.18014,-93.87746"))); didn't run correctly for me, I'm not sure why should it run. I didn't touch that line but that test doesn't run for me.
          Martijn van Groningen made changes -
          Attachment SOLR-2665.patch [ 12487907 ]
          Hide
          Martijn van Groningen added a comment -

          Added initial patch with basic test.
          You can enable post grouping facets and post grouping statistics by using the following parameter:
          group.after=true|false
          The default is false.

          Better names are welcome. I initially wanted to name it group.groupBasedDocSet. Because the DocSet used by faceting and statistics is based on groups.

          The docset is computed based on the first field / func command.

          Show
          Martijn van Groningen added a comment - Added initial patch with basic test. You can enable post grouping facets and post grouping statistics by using the following parameter: group.after=true|false The default is false. Better names are welcome. I initially wanted to name it group.groupBasedDocSet. Because the DocSet used by faceting and statistics is based on groups. The docset is computed based on the first field / func command.
          Martijn van Groningen made changes -
          Link This issue is blocked by LUCENE-3097 [ LUCENE-3097 ]
          Martijn van Groningen made changes -
          Link This issue depends on LUCENE-3097 [ LUCENE-3097 ]
          Martijn van Groningen made changes -
          Assignee Martijn van Groningen [ martijn.v.groningen ]
          Fix Version/s 3.4 [ 12316683 ]
          Fix Version/s 4.0 [ 12314992 ]
          Bill Bell made changes -
          Link This issue relates to SOLR-2524 [ SOLR-2524 ]
          Bill Bell made changes -
          Field Original Value New Value
          Link This issue is blocked by LUCENE-3097 [ LUCENE-3097 ]
          Bill Bell created issue -

            People

            • Assignee:
              Martijn van Groningen
              Reporter:
              Bill Bell
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development