[CURATOR-640] IllegalThreadStateException thrown when access PathChildrenCache in different thread groups

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 5.2.1
• Fix Version/s: 5.3.0
• Component/s: Recipes
• Assignee: Zili Chen
• Reporter: Xiaogang Shi
• Created: 27/May/22 15:17
• Updated: 18/Jul/22 15:46
• Resolved: 17/Jun/22 08:01

Description

When we create PathChildrenCache instances without specifying thead factory, the defaultThreadFactory will be used to create threads. The defaultThreadFactory is typically backed by java.util.concurrent.DefaultThreadFactory which will add newly created threads into the thread group where it is instantiated. That is, the defaultThreadFactory will always add newly created threads into the first thread group that loads the PathChildrenCache class.

In cases where PathChildrenCache is accessed in different thread groups, if the first thread group that load the PathChildrenCache is destroy, then all attempts to start the PathChildrenCache will fail.

The problem can be reproduced using the following code:

ThreadGroup threadGroup1 = new ThreadGroup("testGroup1");
Thread thread1 = new Thread(threadGroup1, () -> {
        PathChildrenCache cache1 =
                new PathChildrenCache(client, "/test1", true);
        try {
            cache1.start();
            Thread.sleep(1000);
        } catch (Throwable t) {
            t.printStackTrace();
        } finally {
            try {
                cache1.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
});

thread1.start();
thread1.join();

// After the thread group is destroyed, all PathChildrenCache instances 
// will fail to start due to inability to add new threads into the first thread group
threadGroup1.destroy();

ThreadGroup threadGroup2 = new ThreadGroup("testGroup2");
Thread thread2 = new Thread(threadGroup2, () -> {
        PathChildrenCache cache2 =
            new PathChildrenCache(client, "/test1", true);
        try {
            cache2.start();
            Thread.sleep(1000);
        } catch (Throwable t) {
            t.printStackTrace();
        } finally {
            try {
                cache2.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
});

thread2.start();
thread2.join();

When starting cache2, the following exception is thrown:

java.lang.IllegalThreadStateException
	at java.base/java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:892)
	at java.base/java.lang.Thread.<init>(Thread.java:434)
	at java.base/java.lang.Thread.<init>(Thread.java:708)
	at java.base/java.util.concurrent.Executors$DefaultThreadFactory.newThread(Executors.java:660)
	at org.apache.curator.shaded.com.google.common.util.concurrent.ThreadFactoryBuilder$1.newThread(ThreadFactoryBuilder.java:163)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.<init>(ThreadPoolExecutor.java:630)
	at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:920)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1353)
	at java.base/java.util.concurrent.Executors$DelegatedExecutorService.execute(Executors.java:721)
	at org.apache.curator.utils.CloseableExecutorService.submit(CloseableExecutorService.java:191)
	at org.apache.curator.framework.recipes.cache.PathChildrenCache.submitToExecutor(PathChildrenCache.java:841)
	at org.apache.curator.framework.recipes.cache.PathChildrenCache.offerOperation(PathChildrenCache.java:792)
	at org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:300)
	at org.apache.curator.framework.recipes.cache.PathChildrenCache.start(PathChildrenCache.java:239)