Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-443

ConjunctionScorer tune-up

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.9
    • 2.1
    • core/search
    • None
    • Linux, Java 1.5, Large Index with 4 million items and some heavily nested boolean queries

    Description

      I just recently ran a load test on the latest code from lucene , which is using a new BooleanScore and noticed the ConjunctionScorer was crunching through objects , especially while sorting as part of the skipTo call. It turns a linked list into an array, sorts the array, then converts the array back to a linked list for further processing by the scoring engines below.

      'm not sure if anyone else is experiencing this as I have a very large index (> 4 million items) and I am issuing some heavily nested queries

      Anyway, I decide to change the link list into an array and use a first and last marker to "simulate" a linked list.

      This scaled much better during my load test as the java gargbage collector was less - umm - virulent

      Attachments

        1. ConjunctionScorer.java
          4 kB
          Abdul Chaudhry
        2. ConjunctionScorer.java
          4 kB
          Paul Elschot
        3. Conjunction20060921.patch
          5 kB
          Paul Elschot

        Activity

          People

            yseeley@gmail.com Yonik Seeley
            abdollar Abdul Chaudhry
            Votes:
            7 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: