Lucene - Core
  1. Lucene - Core
  2. LUCENE-6772

MultiCollector should catch CollectionTerminatedException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      If you wrap two collectors in a MultiCollector and one of them terminates early, then it will also make the other one terminate since MultiCollector propagates the CollectionTerminatedException.

        Issue Links

          Activity

          Hide
          Adrien Grand added a comment -

          Here is a patch.

          I also verified that it did not hurt performance by modifying luceneutil a bit so that it runs two collectors, and ran on wikimedium1m. This change doesn't make any difference:

          diff --git a/src/main/perf/SearchTask.java b/src/main/perf/SearchTask.java
          index 188638d..d46071c 100644
          --- a/src/main/perf/SearchTask.java
          +++ b/src/main/perf/SearchTask.java
          @@ -32,7 +32,7 @@ import org.apache.lucene.facet.range.LongRangeFacetCounts;
           import org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts;
           import org.apache.lucene.index.StorableField;
           import org.apache.lucene.index.StoredDocument;
          -import org.apache.lucene.search.Collector;
          +import org.apache.lucene.search.*;
           import org.apache.lucene.search.FieldDoc;
           import org.apache.lucene.search.IndexSearcher;
           import org.apache.lucene.search.Query;
          @@ -221,7 +221,11 @@ final class SearchTask extends Task {
                     getFacetResultsMsec = (System.nanoTime() - t0)/1000000.0;
                   }
                 } else if (s == null) {
          -        hits = searcher.search(q, topN);
          +        TotalHitCountCollector c1 = new TotalHitCountCollector();
          +        TopScoreDocCollector c2 = TopScoreDocCollector.create(10);
          +        searcher.search(q, MultiCollector.wrap(c1, c2));
          +        hits = c2.topDocs();
          +        //hits = searcher.search(q, topN);
                   if (doHilite) {
                     hilite(hits, state, searcher, q);
                   }
          
                              TaskQPS baseline      StdDev   QPS patch      StdDev                Pct diff
                            Fuzzy1      130.31     (17.1%)      121.84     (17.1%)   -6.5% ( -34% -   33%)
                        AndHighMed      745.89      (6.7%)      730.00      (6.4%)   -2.1% ( -14% -   11%)
                       AndHighHigh      188.09      (2.7%)      184.12      (5.0%)   -2.1% (  -9% -    5%)
                            Fuzzy2       41.15     (12.6%)       40.34     (12.7%)   -2.0% ( -24% -   26%)
                        AndHighLow     1621.12      (8.5%)     1594.15      (8.7%)   -1.7% ( -17% -   16%)
                           Prefix3      441.43     (10.4%)      434.16      (9.7%)   -1.6% ( -19% -   20%)
                           Respell      146.15      (8.7%)      144.38      (8.4%)   -1.2% ( -16% -   17%)
                          HighTerm      567.69      (5.9%)      561.25      (7.7%)   -1.1% ( -13% -   13%)
                       MedSpanNear      115.47      (4.3%)      114.23      (4.7%)   -1.1% (  -9% -    8%)
                         OrHighMed      310.01      (7.5%)      307.18      (7.3%)   -0.9% ( -14% -   14%)
                         LowPhrase     1915.84      (7.6%)     1911.45      (5.2%)   -0.2% ( -12% -   13%)
                          MatchAll      123.15      (2.3%)      122.95      (3.5%)   -0.2% (  -5% -    5%)
                         OrHighLow      190.67      (7.3%)      190.43      (8.9%)   -0.1% ( -15% -   17%)
                            IntNRQ       45.37      (5.4%)       45.37      (5.6%)    0.0% ( -10% -   11%)
                      HighSpanNear      107.41      (8.9%)      107.47      (4.2%)    0.1% ( -12% -   14%)
                           LowTerm     3068.58     (12.6%)     3074.91      (9.4%)    0.2% ( -19% -   25%)
                  HighSloppyPhrase       97.20      (4.7%)       97.49      (5.0%)    0.3% (  -9% -   10%)
                          Wildcard      549.64      (9.8%)      551.62      (9.7%)    0.4% ( -17% -   21%)
                        OrHighHigh      220.99      (9.1%)      221.84      (7.2%)    0.4% ( -14% -   18%)
                           MedTerm     1210.22      (6.4%)     1216.69      (6.5%)    0.5% ( -11% -   14%)
                       LowSpanNear      383.33      (9.9%)      386.49      (7.7%)    0.8% ( -15% -   20%)
                   MedSloppyPhrase      210.81      (8.0%)      212.89      (3.9%)    1.0% ( -10% -   13%)
                   LowSloppyPhrase      332.48      (6.2%)      337.41      (4.7%)    1.5% (  -8% -   13%)
                         MedPhrase      300.05      (6.6%)      305.93      (6.2%)    2.0% ( -10% -   15%)
                        HighPhrase      119.23      (5.3%)      122.03      (5.1%)    2.3% (  -7% -   13%)
          
          Show
          Adrien Grand added a comment - Here is a patch. I also verified that it did not hurt performance by modifying luceneutil a bit so that it runs two collectors, and ran on wikimedium1m. This change doesn't make any difference: diff --git a/src/main/perf/SearchTask.java b/src/main/perf/SearchTask.java index 188638d..d46071c 100644 --- a/src/main/perf/SearchTask.java +++ b/src/main/perf/SearchTask.java @@ -32,7 +32,7 @@ import org.apache.lucene.facet.range.LongRangeFacetCounts; import org.apache.lucene.facet.taxonomy.FastTaxonomyFacetCounts; import org.apache.lucene.index.StorableField; import org.apache.lucene.index.StoredDocument; -import org.apache.lucene.search.Collector; +import org.apache.lucene.search.*; import org.apache.lucene.search.FieldDoc; import org.apache.lucene.search.IndexSearcher; import org.apache.lucene.search.Query; @@ -221,7 +221,11 @@ final class SearchTask extends Task { getFacetResultsMsec = (System.nanoTime() - t0)/1000000.0; } } else if (s == null) { - hits = searcher.search(q, topN); + TotalHitCountCollector c1 = new TotalHitCountCollector(); + TopScoreDocCollector c2 = TopScoreDocCollector.create(10); + searcher.search(q, MultiCollector.wrap(c1, c2)); + hits = c2.topDocs(); + //hits = searcher.search(q, topN); if (doHilite) { hilite(hits, state, searcher, q); } TaskQPS baseline StdDev QPS patch StdDev Pct diff Fuzzy1 130.31 (17.1%) 121.84 (17.1%) -6.5% ( -34% - 33%) AndHighMed 745.89 (6.7%) 730.00 (6.4%) -2.1% ( -14% - 11%) AndHighHigh 188.09 (2.7%) 184.12 (5.0%) -2.1% ( -9% - 5%) Fuzzy2 41.15 (12.6%) 40.34 (12.7%) -2.0% ( -24% - 26%) AndHighLow 1621.12 (8.5%) 1594.15 (8.7%) -1.7% ( -17% - 16%) Prefix3 441.43 (10.4%) 434.16 (9.7%) -1.6% ( -19% - 20%) Respell 146.15 (8.7%) 144.38 (8.4%) -1.2% ( -16% - 17%) HighTerm 567.69 (5.9%) 561.25 (7.7%) -1.1% ( -13% - 13%) MedSpanNear 115.47 (4.3%) 114.23 (4.7%) -1.1% ( -9% - 8%) OrHighMed 310.01 (7.5%) 307.18 (7.3%) -0.9% ( -14% - 14%) LowPhrase 1915.84 (7.6%) 1911.45 (5.2%) -0.2% ( -12% - 13%) MatchAll 123.15 (2.3%) 122.95 (3.5%) -0.2% ( -5% - 5%) OrHighLow 190.67 (7.3%) 190.43 (8.9%) -0.1% ( -15% - 17%) IntNRQ 45.37 (5.4%) 45.37 (5.6%) 0.0% ( -10% - 11%) HighSpanNear 107.41 (8.9%) 107.47 (4.2%) 0.1% ( -12% - 14%) LowTerm 3068.58 (12.6%) 3074.91 (9.4%) 0.2% ( -19% - 25%) HighSloppyPhrase 97.20 (4.7%) 97.49 (5.0%) 0.3% ( -9% - 10%) Wildcard 549.64 (9.8%) 551.62 (9.7%) 0.4% ( -17% - 21%) OrHighHigh 220.99 (9.1%) 221.84 (7.2%) 0.4% ( -14% - 18%) MedTerm 1210.22 (6.4%) 1216.69 (6.5%) 0.5% ( -11% - 14%) LowSpanNear 383.33 (9.9%) 386.49 (7.7%) 0.8% ( -15% - 20%) MedSloppyPhrase 210.81 (8.0%) 212.89 (3.9%) 1.0% ( -10% - 13%) LowSloppyPhrase 332.48 (6.2%) 337.41 (4.7%) 1.5% ( -8% - 13%) MedPhrase 300.05 (6.6%) 305.93 (6.2%) 2.0% ( -10% - 15%) HighPhrase 119.23 (5.3%) 122.03 (5.1%) 2.3% ( -7% - 13%)
          Hide
          ASF subversion and git services added a comment -

          Commit 1701231 from Adrien Grand in branch 'dev/trunk'
          [ https://svn.apache.org/r1701231 ]

          LUCENE-6772: MultiCollector now handles CollectionTerminatedException.

          Show
          ASF subversion and git services added a comment - Commit 1701231 from Adrien Grand in branch 'dev/trunk' [ https://svn.apache.org/r1701231 ] LUCENE-6772 : MultiCollector now handles CollectionTerminatedException.
          Hide
          ASF subversion and git services added a comment -

          Commit 1701233 from Adrien Grand in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1701233 ]

          LUCENE-6772: MultiCollector now handles CollectionTerminatedException.

          Show
          ASF subversion and git services added a comment - Commit 1701233 from Adrien Grand in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1701233 ] LUCENE-6772 : MultiCollector now handles CollectionTerminatedException.

            People

            • Assignee:
              Adrien Grand
              Reporter:
              Adrien Grand
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development