[SOLR-16187] ExecutorUtil#awaitTermination shouldn't wait forever

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 9.1, main (10.0), 9.0.1
• Assignee: Kevin Risden
• Reporter: Kevin Risden
• Created: 05/May/22 20:04
• Updated: 30/Apr/24 16:19
• Resolved: 06/Sep/22 14:35

Description

While looking at some of the other thread leak issues, found that ExecutorUtil#awaitTermination is waiting basically forever:

https://github.com/apache/solr/blob/main/solr/solrj/src/java/org/apache/solr/common/util/ExecutorUtil.java#L98

public static void awaitTermination(ExecutorService pool) {
    boolean shutdown = false;
    while (!shutdown) {
      try {
        // Wait a while for existing tasks to terminate
        shutdown = pool.awaitTermination(60, TimeUnit.SECONDS);
      } catch (InterruptedException ie) {
        // Preserve interrupt status
        Thread.currentThread().interrupt();
      }
    }
  }

Instead it should be possible to wait and then if still not shutdown then interrupt the threads? ie: pool.shutdownNow() then another awaitTermination to at least limit the termination time to 2*60 seconds?

This would at least have some bound to shutting stuff down.

Issue Links

• causes: SOLR-17261 Core loading is capped at 60 seconds
• causes: SOLR-17000 ExecutorUtilTest failures due to bad concurrency assumptions in test logic
• is related to: SOLR-16401 Test thread leak linger should be 1s
• relates to: SOLR-16154 ZKEventListenerThread leaks from tests
• links to: GitHub Pull Request #840