[CASSANDRA-3494] Streaming is mono-threaded (the bulk loader too by extension)

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 1.1.0
• Assignee: Peter Schuller
• Reporter: Sylvain Lebresne

Authors

Peter Schuller

Reviewers

Yuki Morishita

• Created: 15/Nov/11 16:40
• Updated: 16/Apr/19 09:32
• Resolved: 06/Dec/11 02:23

Description

The streamExecutor is define as:

streamExecutor_ = new DebuggableThreadPoolExecutor("Streaming", Thread.MIN_PRIORITY);

In the meantime, in DebuggableThreadPoolExecutor.java:

public DebuggableThreadPoolExecutor(String threadPoolName, int priority)
{
   this(1, Integer.MAX_VALUE, TimeUnit.SECONDS, new LinkedBlockingQueue<Runnable>(), new NamedThreadFactory(threadPoolName, priority));
}

In other word, since the core pool size is 1 and the queue unbounded, tasks will always queued and the executor is essentially mono-threaded.
This is clearly not necessary since we already have stream throttling nowadays. And it could be a limiting factor in the case of the bulk loader.

Besides, I would venture that this maybe was not the intention, because putting the max core size to MAX_VALUE would suggest that the intention was to spawn threads on demand.

Issue Links

• is duplicated by: CASSANDRA-3549 streaming is hard-coded to be single-threaded