Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-8464

Support direct buffer decompression for reads

    XMLWordPrintableJSON

    Details

      Description

      Currently when we read a compressed sstable we copy the data on heap then send it to be de-compressed to another on heap buffer (albeit pooled).

      But now both snappy and lz4 (with CASSANDRA-7039) allow decompression of direct byte buffers. This lets us mmap the data and decompress completely off heap (and avoids moving bytes over JNI).

      One issue is performing the checksum offheap but the Adler32 does support in java 8 (it's also in java 7 but marked private?!)

      This change yields a > 10% boost in read performance on cstar. Locally I see upto 30% improvement.

      http://cstar.datastax.com/graph?stats=5ebcdd70-816b-11e4-aed6-42010af0688f&metric=op_rate&operation=2_read&smoothing=1&show_aggregates=true&xmin=0&xmax=200.09&ymin=0&ymax=135908.3

        Attachments

        1. compression_direct.png
          100 kB
          T Jake Luciani

          Issue Links

            Activity

              People

              • Assignee:
                tjake T Jake Luciani
                Reporter:
                tjake T Jake Luciani
                Authors:
                T Jake Luciani
                Reviewers:
                Branimir Lambov
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: