Uploaded image for project: 'Apache Cassandra'
  1. Apache Cassandra
  2. CASSANDRA-4310

Multiple independent Level Compactions in Parallel

    XMLWordPrintableJSON

Details

    Description

      Problem: If you are inserting data into cassandra and level compaction cannot catchup, you will create lot of files in L0.

      Here is a solution which will help here and also increase the performance of level compaction.

      We can do many compactions in parallel for unrelated data.
      1) For no over lapping levels. Ex: when L0 stable is compacting with L1, we can do compactions in other levels like L2 and L3 if they are eligible.
      2) We can also do compactions with files in L1 which are not participating in L0 compactions.

      This is specially useful if you are using SSD and is not bottlenecked by IO.

      I am seeing this issue in my cluster. The compactions pending are more than 50k and the disk usage is not that much(I am using SSD).
      I am doing multithreaded to true and also not throttling the IO by putting the value as 0.

      Attachments

        1. 4310.txt
          25 kB
          Yuki Morishita
        2. 4310-v2.txt
          27 kB
          Yuki Morishita
        3. 4310-v3.txt
          33 kB
          Yuki Morishita
        4. 4310-v5.txt
          54 kB
          Jonathan Ellis
        5. 4310-v6.txt
          55 kB
          Yuki Morishita

        Activity

          People

            yukim Yuki Morishita
            kohlisankalp Sankalp Kohli
            Yuki Morishita
            Jonathan Ellis
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: