Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-5773

CollapsingQParserPlugin should make elevated documents the group head

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.6.1
    • Fix Version/s: 4.8
    • Component/s: query parsers
    • Labels:

      Description

      Hi Joel,
      I sent you an email but I'm not sure if you received it or not. I ran into a bit of trouble using the CollapsingQParserPlugin with elevated documents. To explain it simply, I want to exclude grouped documents when one of the members of the group are contained in the elevated document set. I'm not sure this is possible currently because as you explain above elevated documents are added to the request context after the original query is constructed.

      To try to better illustrate the problem. If I have 2 documents docid=1 and docid=2 and both have a groupid of 'a'. If a grouped query scores docid 2 first in the results but I have elevated docid 1 then both documents are shown in the results when I really only want the elevated document to be shown in the results.
      Is this something that would be difficult to implement? Any help is appreciated.

      I think the solution would be to remove the documents from liveDocs that share the same groupid in the getBoostDocs() function. Let me know if this makes any sense. I'll continue working towards a solution in the meantime.

      private IntOpenHashSet getBoostDocs(SolrIndexSearcher indexSearcher, Set<String> boosted) throws IOException {
            IntOpenHashSet boostDocs = null;
            if(boosted != null) {
              SchemaField idField = indexSearcher.getSchema().getUniqueKeyField();
              String fieldName = idField.getName();
              HashSet<BytesRef> localBoosts = new HashSet(boosted.size()*2);
              Iterator<String> boostedIt = boosted.iterator();
              while(boostedIt.hasNext()) {
                localBoosts.add(new BytesRef(boostedIt.next()));
              }
      
              boostDocs = new IntOpenHashSet(boosted.size()*2);
      
              List<AtomicReaderContext>leaves = indexSearcher.getTopReaderContext().leaves();
              TermsEnum termsEnum = null;
              DocsEnum docsEnum = null;
              for(AtomicReaderContext leaf : leaves) {
                AtomicReader reader = leaf.reader();
                int docBase = leaf.docBase;
                Bits liveDocs = reader.getLiveDocs();
                Terms terms = reader.terms(fieldName);
                termsEnum = terms.iterator(termsEnum);
                Iterator<BytesRef> it = localBoosts.iterator();
                while(it.hasNext()) {
                  BytesRef ref = it.next();
                  if(termsEnum.seekExact(ref)) {
                    docsEnum = termsEnum.docs(liveDocs, docsEnum);
                    int doc = docsEnum.nextDoc();
                    if(doc != -1) {
                      //Found the document.
                      boostDocs.add(doc+docBase);
      
                     *// HERE REMOVE ANY DOCUMENTS THAT SHARE THE GROUPID NOT ONLY THE DOCID //*
                      it.remove();
      
      
      
                    }
                  }
                }
              }
            }
      
            return boostDocs;
          }
      

        Attachments

        1. SOLR-5773.patch
          7 kB
          David Boychuck
        2. SOLR-5773.patch
          13 kB
          David Boychuck
        3. SOLR-5773.patch
          10 kB
          Joel Bernstein
        4. SOLR-5773.patch
          11 kB
          Joel Bernstein
        5. SOLR-5773.patch
          11 kB
          Joel Bernstein
        6. SOLR-5773.patch
          12 kB
          Joel Bernstein
        7. SOLR-5773.patch
          13 kB
          Joel Bernstein

          Issue Links

            Activity

              People

              • Assignee:
                joel.bernstein Joel Bernstein
                Reporter:
                dboychuck David Boychuck
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 8h
                  8h
                  Remaining:
                  Remaining Estimate - 8h
                  8h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified