[LUCENE-9066] Modal Strategy In Concurrent Query Execution

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Atri Sharma
• Created: 26/Nov/19 07:33
• Updated: 28/Aug/22 15:53

Description

When executing a query concurrently today, we do not take any sort of system statistics into account. For e.g. if the node is under high pressure, it is not advisable to spawn a large number of threads for a query since they are most likely to be blocked waiting for CPU to be available. However, the converse is that for a lightly loaded cluster, the query can consume as many threads as required.

 

This Jira tracks high level efforts in this direction. The first idea is to account the Executor's wait queue's size as a factor when allocating slices to a query's segments.

Issue Links

• is a parent of: LUCENE-9075 Check Current System Pressure Before Allocating a New Thread In Concurrent Search
• is a parent of: LUCENE-9074 Account for Executor's Queue Length When Planning Slices in IndexSearcher