[LUCENE-1309] Using IndexSearcher in a multithreaded context causes contention

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.3.1, 2.3.2

Environment

• WinXP, Dual Core Centrino
• Opensuse Linux, Dual Core AMD

java version "1.6.0_06"
Java(TM) SE Runtime Environment (build 1.6.0_06-b02)
Java HotSpot(TM) Client VM (build 10.0-b22, mixed mode, sharing)

• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Rene Schwietzke
• Created: 18/Jun/08 19:59
• Updated: 28/Aug/22 11:50
• Resolved: 17/Nov/08 10:29

Description

The attached code allows to search with a single IndexSearcher across threads or with a single instance per Thread, additionally it allows to deals with String.intern in Field from LUCENE-1308.

Really interesting is the behavior of the current implementation (Field String.intern) The performance improves dramatically.

Is this by design? Because it is usually recommended to reuse the IndexSearcher. How about a version to supports mult-threading much better basically delivering a read-only view?

The following results have been measured. The [main] part is a single threaded run. The following thread results

Field String.intern, Single Searcher
[main] Search took: 3453ms
[Thread-2] Search took: 17812ms
[Thread-3] Search took: 18313ms
[Thread-1] Search took: 18234ms
[Thread-0] Search took: 18562ms

Field WeakHashMap, Single Searcher
[main] Search took: 3156ms
[Thread-3] Search took: 14953ms
[Thread-1] Search took: 15593ms
[Thread-0] Search took: 15656ms
[Thread-2] Search took: 16188ms

Field ConcurrentHashMap, Single Searcher
[main] Search took: 2844ms
[Thread-1] Search took: 14812ms
[Thread-0] Search took: 14890ms
[Thread-2] Search took: 15172ms
[Thread-3] Search took: 14656ms

Field intern, Searcher per Thread
[main] Search took: 3687ms
[Thread-2] Search took: 10766ms
[Thread-1] Search took: 10734ms
[Thread-0] Search took: 11047ms
[Thread-3] Search took: 10938ms

Field Weak, Searcher per Thread
[main] Search took: 3187ms
[Thread-3] Search took: 11625ms
[Thread-2] Search took: 12484ms
[Thread-1] Search took: 12609ms
[Thread-0] Search took: 12391ms

Field Concurrent, Searcher per Thread
[main] Search took: 2938ms
[Thread-3] Search took: 10203ms
[Thread-2] Search took: 10360ms
[Thread-1] Search took: 10406ms
[Thread-0] Search took: 10719ms

Issue Links

• is related to: LUCENE-1308 Remove String.intern() from Field.java to increase performance and lower contention
• is related to: LUCENE-1265 Identify bottleneck associated with not pooling searchers