Lucene - Core
  1. Lucene - Core
  2. LUCENE-3503

DisjunctionSumScorer gives slightly (float iotas) different scores when you .nextDoc vs .advance

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      Spinoff from LUCENE-1536.

      I dug into why we hit a score diff when using luceneutil to benchmark
      the patch.

      At first I thought it was BS1/BS2 difference, but because of a bug in
      the patch it was still using BS2 (but should be BS1) – Robert's last
      patch fixes that.

      But it's actually a diff in BS2 itself, whether you next or advance
      through the docs.

      It's because DisjunctionSumScorer, when summing the float scores for a
      given doc that matches multiple sub-scorers, might sum in a different
      order, when you had .nextDoc'd to that doc than when you had .advance'd
      to it.

      This in turn is because the PQ used by that scorer (ScorerDocQueue)
      makes no effort to break ties. So, when the top N scorers are on the
      same doc, the PQ doesn't care what order they are in.

      Fixing ScorerDocQueue to break ties will likely be a non-trivial perf
      hit, though, so I'm not sure whether we should do anything here...

      1. LUCENE-3503.patch
        5 kB
        Robert Muir
      2. LUCENE-3503.patch
        5 kB
        Robert Muir
      3. LUCENE-3503.patch
        4 kB
        Michael McCandless

        Issue Links

          Activity

          Michael McCandless created issue -
          Hide
          Michael McCandless added a comment -

          Failing test case showing the bug.

          Show
          Michael McCandless added a comment - Failing test case showing the bug.
          Michael McCandless made changes -
          Field Original Value New Value
          Attachment LUCENE-3503.patch [ 12498442 ]
          Hide
          Robert Muir added a comment -

          I think we have to fix it, get the correctness and then worry about performance later.

          giving a document a different score (no matter how little, it will affect ranking) just because you next'ed versus advance'd it is bad news.

          Show
          Robert Muir added a comment - I think we have to fix it, get the correctness and then worry about performance later. giving a document a different score (no matter how little, it will affect ranking) just because you next'ed versus advance'd it is bad news.
          Robert Muir made changes -
          Link This issue blocks LUCENE-1536 [ LUCENE-1536 ]
          Hide
          Robert Muir added a comment -

          not a real 'fix' but maybe this solves it for practical purposes?

          Show
          Robert Muir added a comment - not a real 'fix' but maybe this solves it for practical purposes?
          Robert Muir made changes -
          Attachment LUCENE-3503.patch [ 12498447 ]
          Hide
          Robert Muir added a comment -

          patch with a bugfix to the test (in case it gets slowmultireaderwrapper)

          Show
          Robert Muir added a comment - patch with a bugfix to the test (in case it gets slowmultireaderwrapper)
          Robert Muir made changes -
          Attachment LUCENE-3503.patch [ 12498453 ]
          Robert Muir made changes -
          Assignee Robert Muir [ rcmuir ]
          Robert Muir made changes -
          Fix Version/s 3.5 [ 12317877 ]
          Fix Version/s 4.0 [ 12314025 ]
          rmuir committed 1181659 (3 files)
          Robert Muir made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Uwe Schindler added a comment -

          Bulk close after release of 3.5

          Show
          Uwe Schindler added a comment - Bulk close after release of 3.5
          Uwe Schindler made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Robert Muir
              Reporter:
              Michael McCandless
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development