[CASSANDRA-6809] Compressed Commit Log

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.2.0 beta 1
• Labels: performance qa-resolved
• Impacts: Docs
• Assignee: Branimir Lambov
• Reporter: Benedict Elliott Smith

Authors

Branimir Lambov

Reviewers

Ariel Weisberg

• Tester: Alan Boudreault
• Created: 06/Mar/14 19:54
• Updated: 06/Nov/19 22:05
• Resolved: 25/Mar/15 12:18

Description

It seems an unnecessary oversight that we don't compress the commit log. Doing so should improve throughput, but some care will need to be taken to ensure we use as much of a segment as possible. I propose decoupling the writing of the records from the segments. Basically write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the compressed chunks into a CLS as possible.

Issue Links

• blocks: CASSANDRA-6018 Add option to encrypt commitlog
• blocks: CASSANDRA-8771 Remove commit log segment recycling
• is depended upon by: CASSANDRA-8634 Multi-thread commit log compression
• is duplicated by: CASSANDRA-7994 Commit logs on the fly compression
• is duplicated by: CASSANDRA-4148 Explore commitlog compression
• is related to: CASSANDRA-8729 Commitlog causes read before write when overwriting
• is related to: CASSANDRA-9096 Improve ByteBuffer compression interface
• is related to: CASSANDRA-9095 Compressed commit log should measure compressed space used