[LUCENE-10576] ConcurrentMergeScheduler maxThreadCount calculation is artificially low

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Lucene Fields: New
• Assignee: Kevin Risden
• Reporter: Kevin Risden
• Created: 16/May/22 17:44
• Updated: 28/Aug/22 16:41
• Resolved: 17/May/22 12:50

Description

https://github.com/apache/lucene/blob/main/lucene/core/src/java/org/apache/lucene/index/ConcurrentMergeScheduler.java#L177

maxThreadCount = Math.max(1, Math.min(4, coreCount / 2));

This has a practical limit of max of 4 threads due to the Math.min. This doesn't take into account higher coreCount.

I can't seem to tell if this is by design or this is just a mix up of logic during the calculation.

If I understand it looks like 1 and 4 are mixed up and should instead be:

maxThreadCount = Math.max(4, Math.min(1, coreCount / 2));

which then simplifies to

maxThreadCount = Math.max(4, coreCount / 2);

So that you have a minimum of 4 maxThreadCount and max of coreCount/2.

---

Based on the history I could find, this has been this way forever.

• LUCENE-6437
• LUCENE-6119
• LUCENE-5951
  • Introduced as "maxThreadCount = Math.max(1, Math.min(3, Runtime.getRuntime().availableProcessors()/2));"
  • https://github.com/apache/lucene/commit/33410e30c1af7105a6b8b922255af047d13be626#diff-ceb8ec6fe5807682cfb691a8ec52bcc672fb7c5eeb6922c80da4c075f7f003c8R147

Issue Links

• relates to: LUCENE-5951 Detect when index is on SSD and set dynamic defaults
• relates to: LUCENE-6119 Add auto-io-throttle to ConcurrentMergeScheduler
• relates to: LUCENE-6437 Improve test reproducibility of ConcurrentMergeScheduler
• links to: GitHub Pull Request #895