[SOLR-11208] Usage SynchronousQueue in Executors prevent large scale operations

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 6.6
• Assignee: Unassigned
• Reporter: Björn Häuser
• Created: 07/Aug/17 19:02
• Updated: 16/Sep/20 22:38

Description

I am not sure where to start with this one.

I tried to post this already on the mailing list: https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201708.mbox/[e-mail]%3e

In short: the usage of a SynchronousQueue as the workQeue prevents more tasks than max threads.

For example, taken from OverseerCollectionMessageHandler:

  ExecutorService tpe = new ExecutorUtil.MDCAwareThreadPoolExecutor(5, 10, 0L, TimeUnit.MILLISECONDS,
      new SynchronousQueue<>(),
      new DefaultSolrThreadFactory("OverseerCollectionMessageHandlerThreadFactory"));

This Executor is used when doing a REPLACENODE (= ADDREPLICA) command. When the node has more than 10 collections this will fail with the mentioned java.util.concurrent.RejectedExecutionException.

I am also not sure how to fix this. Just replacing the queue with a different implementation feels wrong to me or could cause unwanted side behaviour.

Thanks

Issue Links

• links to: GitHub Pull Request #1664