Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-443

ConjunctionScorer tune-up

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.9
    • Fix Version/s: 2.1
    • Component/s: core/search
    • Labels:
      None
    • Environment:

      Linux, Java 1.5, Large Index with 4 million items and some heavily nested boolean queries

      Description

      I just recently ran a load test on the latest code from lucene , which is using a new BooleanScore and noticed the ConjunctionScorer was crunching through objects , especially while sorting as part of the skipTo call. It turns a linked list into an array, sorts the array, then converts the array back to a linked list for further processing by the scoring engines below.

      'm not sure if anyone else is experiencing this as I have a very large index (> 4 million items) and I am issuing some heavily nested queries

      Anyway, I decide to change the link list into an array and use a first and last marker to "simulate" a linked list.

      This scaled much better during my load test as the java gargbage collector was less - umm - virulent

        Attachments

        1. Conjunction20060921.patch
          5 kB
          Paul Elschot
        2. ConjunctionScorer.java
          4 kB
          Paul Elschot
        3. ConjunctionScorer.java
          4 kB
          Abdul Chaudhry

          Activity

            People

            • Assignee:
              yseeley@gmail.com Yonik Seeley
              Reporter:
              abdollar Abdul Chaudhry
            • Votes:
              7 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: