[CASSANDRA-7139] Default concurrent_compactors is probably too high

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.1 rc1
• Assignee: Jonathan Ellis
• Reporter: Benedict Elliott Smith

Authors

Jonathan Ellis

Reviewers

Benedict Elliott Smith

• Created: 02/May/14 12:23
• Updated: 16/Apr/19 09:31
• Resolved: 21/May/14 18:11

Description

The default number of concurrent compactors is probably too high for modern hardware with spinning disks for storage: A modern blade can easily have 24+ Cores, which would result in a default of 24 concurrent compactions. This not only increases random IO, it also keeps around a lot of obsoleted files for an unnecessarily long time, as each compaction keeps references to any possibly overlapping files that it isn't itself compacting - but these can have been obsoleted part way through by compactions that finished earlier. If you factor in the default compaction throughput rate of 16Mb/s, anything but a single default concurrent_compactor makes very little sense, as a single thread should always be able to handle 16Mb/s, will cause less interference with other processes, and permits obsoleted files to be immediately removed.

See http://imgur.com/HDqhxFp for a graph demonstrating the result of making this change on a box with 24-cores and 8Tb of storage (first spike is default settings)

Issue Links

• relates to: CASSANDRA-7819 In progress compactions should not prevent deletion of stale sstables