[CASSANDRA-7130] Make sstable checksum type configurable and optional

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Duplicate
• Labels: performance
• Assignee: Jason Brown
• Reporter: Benedict Elliott Smith

Authors

Jason Brown

• Created: 01/May/14 16:33
• Updated: 16/Apr/19 09:31
• Resolved: 24/Feb/15 00:32

Description

A lot of our users are becoming bottlenecked on CPU rather than IO, and whilst Adler32 is faster than CRC, it isn't anything like as fast as xxhash (used by LZ4), which can push Gb/s. I propose making the checksum type configurable so that users who want speed can shift to xxhash, and those who want security can use Adler or CRC.

It's worth noting that at some point in the future (JDK8?) optimised implementations using latest intel crc instructions will be added, though it's not clear from the mailing list discussion if/when that will materialise:

http://mail.openjdk.java.net/pipermail/hotspot-compiler-dev/2013-May/010775.html

Issue Links

• is related to: CASSANDRA-13908 Add JDK9 new CRC32C checksum
• is related to: CASSANDRA-8684 Replace usage of Adler32 with CRC32
• relates to: CASSANDRA-7928 Digest queries do not require alder32 checks