[CASSANDRA-13055] DoS by StreamReceiveTask, during incremental repair

• Type: Bug
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Component/s: Consistency/Repair
• Severity: Normal
• Assignee: Unassigned
• Reporter: Tom van der Woerdt
• Created: 19/Dec/16 11:51
• Updated: 16/Apr/19 09:30

Description

There's no limit on how many StreamReceiveTask there can be, and during an incremental repair on a vnode cluster with high replication factors, this can lead to thousands of conccurent StreamReceiveTask threads, effectively DoSing the node.

I just found one of my nodes with 1000+ loadavg, caused by 1363 concurrent StreamReceiveTask threads.

That sucks

I think :

• Cassandra shouldn't allow more than X concurrent StreamReceiveTask threads
• StreamReceiveTask threads should be at a lower priority, like compaction threads

Alternative ideas welcome as well, of course.

Issue Links

• is related to: CASSANDRA-12965 StreamReceiveTask causing high CPU utilization during repair