Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7994

Commit logs on the fly compression

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Normal
    • Resolution: Duplicate
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      This patch employs lz4 algo to comress commit logs. This could be useful to conserve disk space either archiving commit logs for a long time or for conserviing iops for use cases with often and large mutations updating the same record.

      The compression is performed on blocks of 64k, for better cross mutation compression. CRC is computed on each 64k block, unlike original code computing it on each individual mutation.

      On one of our real production cluster this saved 2/3 of the space consumed by commit logs. The replay is 20-30% slower for the same number of mutations.

      While doing this, also refactored commit log reading code to CommitLogReader class, which i believe makes code cleaner.

        Attachments

        1. CompressedCommitLogs-7994.txt
          74 kB
          Oleg Anastasyev

          Issue Links

            Activity

              People

              • Assignee:
                m0nstermind Oleg Anastasyev
                Reporter:
                m0nstermind Oleg Anastasyev
                Authors:
                Oleg Anastasyev
                Reviewers:
                Jason Brown
              • Votes:
                1 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: