[CASSANDRA-3711] Unsustainable Thread Accumulation in ParallelCompactionIterable.Reducer ThreadPoolExecutor

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 1.0.7
• Labels: compaction memory_leak threading threads

Environment

• Linux version 2.6.32-71.29.1.el6.x86_64 ([e-mail]) (gcc version 4.4.4 20100726 (Red Hat 4.4.4-13) (GCC) ) #1 SMP Mon Jun 27 19:49:27 BST 2011
• java version "1.7.0_01" / Java(TM) SE Runtime Environment (build 1.7.0_01-b08) / Java HotSpot(TM) 64-Bit Server VM (build 21.1-b02, mixed mode)

• Severity: Low
• Assignee: Jonathan Ellis
• Reporter: Caleb Rackliffe

Authors

Jonathan Ellis

Reviewers

Sylvain Lebresne

• Created: 08/Jan/12 23:46
• Updated: 16/Apr/19 09:32
• Resolved: 10/Jan/12 08:29

Description

With multithreaded compaction enabled, it looks like Reducer creates a new thread pool for every compaction. These pools seem to just sit around - i.e. "executor.shutdown()" never gets called and the Threads live forever waiting for tasks that will never come. For instance...

Name: CompactionReducer:1
State: TIMED_WAITING on java.util.concurrent.SynchronousQueue$TransferStack@72938aea
Total blocked: 0 Total waited: 1

Stack trace:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226)
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359)
java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1043)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1103)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
java.lang.Thread.run(Thread.java:722)