[#LUCENE-443] ConjunctionScorer tune-up

Lucene - Java

ConjunctionScorer tune-up

Created: 02/Oct/05 02:24 PM Updated: 27/Feb/07 06:10 PM

Return to search

Component/s:

Affects Version/s:

1.9

Fix Version/s:

2.1

Time Tracking:

Not Specified

File Attachments:

File Name	Date Attached	Attached By	Size
Conjunction20060921.patch	2006-09-21 07:17 AM	Paul Elschot	5 kB
ConjunctionScorer.java	2005-10-03 04:22 AM	Paul Elschot	4 kB
ConjunctionScorer.java	2005-10-02 02:32 PM	Abdul Chaudhry	4 kB

Environment:

Linux, Java 1.5, Large Index with 4 million items and some heavily nested boolean queries

Resolution Date:

17/Oct/06 08:51 PM

Description

« Hide

I just recently ran a load test on the latest code from lucene , which is using a new BooleanScore and noticed the ConjunctionScorer was crunching through objects , especially while sorting as part of the skipTo call. It turns a linked list into an array, sorts the array, then converts the array back to a linked list for further processing by the scoring engines below.

'm not sure if anyone else is experiencing this as I have a very large index (> 4 million items) and I am issuing some heavily nested queries

Anyway, I decide to change the link list into an array and use a first and last marker to "simulate" a linked list.

This scaled much better during my load test as the java gargbage collector was less - umm - virulent

All

Comments

Work Log

Change History

Subversion Commits

Sort Order:

Repository	Revision	Date	User	Message
ASF	#465043	Tue Oct 17 20:50:52 UTC 2006	yonik	ConjunctionScorer performance increase: ~~LUCENE-443~~
				Files Changed
				MODIFY /lucene/java/trunk/src/java/org/apache/lucene/search/ConjunctionScorer.java MODIFY /lucene/java/trunk/CHANGES.txt