Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7139

Default concurrent_compactors is probably too high

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Fix Version/s: 2.1 rc1
    • Component/s: None
    • Labels:
      None

      Description

      The default number of concurrent compactors is probably too high for modern hardware with spinning disks for storage: A modern blade can easily have 24+ Cores, which would result in a default of 24 concurrent compactions. This not only increases random IO, it also keeps around a lot of obsoleted files for an unnecessarily long time, as each compaction keeps references to any possibly overlapping files that it isn't itself compacting - but these can have been obsoleted part way through by compactions that finished earlier. If you factor in the default compaction throughput rate of 16Mb/s, anything but a single default concurrent_compactor makes very little sense, as a single thread should always be able to handle 16Mb/s, will cause less interference with other processes, and permits obsoleted files to be immediately removed.

      See http://imgur.com/HDqhxFp for a graph demonstrating the result of making this change on a box with 24-cores and 8Tb of storage (first spike is default settings)

        Attachments

        1. 7139.txt
          2 kB
          Jonathan Ellis

          Issue Links

            Activity

              People

              • Assignee:
                jbellis Jonathan Ellis
                Reporter:
                benedict Benedict
                Reviewer:
                Benedict
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: