Lucene - Core
  1. Lucene - Core
  2. LUCENE-4952

DrillSideways should expose "scoreSubDocsAtOnce" control

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.4, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      I hit this when running a ToParentBlockJoinCollector/Query under
      DrillSideways ... the problem is ToParentBlockJoinCollector.collect
      expects that all sub-scorers are positioned on the docID being
      collected, but DrillSideways sometimes scores with a in-order
      BooleanScorer-like scorer that advances each sub-scorer in chunks
      ... this breaks ToParentBlockJoinCollector.

      This is the same issue as LUCENE-2686, where apps that want to peek at
      the sub-scorers from their collector need those sub-scorers to all be
      "on" the current docID being collected...

      One way to "fix" this would be to switch based on
      Collector.acceptsDocsOutOfOrder() ... but that's really a hack ... it
      only "works" for BooleanQuery because BooleanScorer is an out-of-order
      scorer (well and because we fixed all BS2s to keep sub-scorers
      positioned on the doc being collected in LUCENE-3505).

      But if for example we added MUST clauses back into BooleanScorer
      (which I think we should!) then it could easily score those queries
      in-order. Really we need another boolean (scoreSubDocsAtOnce or
      something) to Weight.scorer... but that's a big change...

      I think for this issue I'll just add an expert protected method to
      DrillSideways that returns this boolean, and an app could subclass to
      override. Apps that "know" they are using queries/collectors like
      ToParentBlockJoinQuery/Collector must subclass and override
      ... DrillSideways already has other expert methods that you subclass &
      override.

      1. LUCENE-4952.patch
        11 kB
        Michael McCandless

        Activity

        Hide
        Michael McCandless added a comment -

        Patch, I think it's ready.

        I added expert protected method scoreSubDocsAtOnce, and fixed
        TestDrillSideways.testRandom to sometimes return true here.

        I added an AssertingSubDocsAtOnceCollector that then verifies that all
        sub-scorers are positioned on the same docID being collected ...

        Show
        Michael McCandless added a comment - Patch, I think it's ready. I added expert protected method scoreSubDocsAtOnce, and fixed TestDrillSideways.testRandom to sometimes return true here. I added an AssertingSubDocsAtOnceCollector that then verifies that all sub-scorers are positioned on the same docID being collected ...
        Hide
        Commit Tag Bot added a comment -

        [trunk commit] mikemccand
        http://svn.apache.org/viewvc?view=revision&revision=1471732

        LUCENE-4952: add method to force DrillSideways to keep all sub-scorers on the doc being collected

        Show
        Commit Tag Bot added a comment - [trunk commit] mikemccand http://svn.apache.org/viewvc?view=revision&revision=1471732 LUCENE-4952 : add method to force DrillSideways to keep all sub-scorers on the doc being collected
        Hide
        Commit Tag Bot added a comment -

        [trunk commit] mikemccand
        http://svn.apache.org/viewvc?view=revision&revision=1471733

        LUCENE-4952: put CHANGES entry in the right place

        Show
        Commit Tag Bot added a comment - [trunk commit] mikemccand http://svn.apache.org/viewvc?view=revision&revision=1471733 LUCENE-4952 : put CHANGES entry in the right place
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] mikemccand
        http://svn.apache.org/viewvc?view=revision&revision=1471735

        LUCENE-4952: add method to force DrillSideways to keep all sub-scorers on the doc being collected

        Show
        Commit Tag Bot added a comment - [branch_4x commit] mikemccand http://svn.apache.org/viewvc?view=revision&revision=1471735 LUCENE-4952 : add method to force DrillSideways to keep all sub-scorers on the doc being collected
        Hide
        Steve Rowe added a comment -

        Bulk close resolved 4.4 issues

        Show
        Steve Rowe added a comment - Bulk close resolved 4.4 issues

          People

          • Assignee:
            Michael McCandless
            Reporter:
            Michael McCandless
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development