Cassandra
  1. Cassandra
  2. CASSANDRA-3389

Evaluate CSLM alternatives for improved cache or GC performance

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Activity

      Hide
      Jonathan Ellis added a comment -

      I think we've invested as much time as makes sense for us at this point. (Whether CSLM isn't as much of a bottleneck as we thought, or CSTM/ST don't help as much w/ G1 as we thought, is kind of moot.)

      Show
      Jonathan Ellis added a comment - I think we've invested as much time as makes sense for us at this point. (Whether CSLM isn't as much of a bottleneck as we thought, or CSTM/ST don't help as much w/ G1 as we thought, is kind of moot.)
      Hide
      Ben Manes added a comment -

      It might be worth using isolated benchmarks:

      • Google Caliper for single-threaded (GetPutBenchmark)
      • Cliff Click's for multi-threaded (PerfHashBenchmark)

      You can hack from the examples below (see README):
      http://code.google.com/p/concurrentlinkedhashmap/source/browse/trunk#trunk%2Fsrc%2Ftest%2Fjava%2Fcom%2Fgooglecode%2Fconcurrentlinkedhashmap%2Fbenchmark

      Show
      Ben Manes added a comment - It might be worth using isolated benchmarks: Google Caliper for single-threaded (GetPutBenchmark) Cliff Click's for multi-threaded (PerfHashBenchmark) You can hack from the examples below (see README): http://code.google.com/p/concurrentlinkedhashmap/source/browse/trunk#trunk%2Fsrc%2Ftest%2Fjava%2Fcom%2Fgooglecode%2Fconcurrentlinkedhashmap%2Fbenchmark
      Hide
      Brandon Williams added a comment -

      No significant difference with G1.

      Show
      Brandon Williams added a comment - No significant difference with G1.
      Hide
      Jonathan Ellis added a comment -

      Can you test these under G1 garbage collector? CSLM is the main reason G1 works poorly for us. (Especially the uses in Memtable.)

      Show
      Jonathan Ellis added a comment - Can you test these under G1 garbage collector? CSLM is the main reason G1 works poorly for us. (Especially the uses in Memtable.)
      Hide
      Jonathan Ellis added a comment -

      We also want to allow multiple clients to append to a wide row, so the write lock design probably doesn't work for us either.

      Show
      Jonathan Ellis added a comment - We also want to allow multiple clients to append to a wide row, so the write lock design probably doesn't work for us either.
      Hide
      Jason Rutherglen added a comment -

      It is probably worth looking at Accumolo's implementation of a NativeMap [1] that implements sorted key value pairs in C++ to avoid 'stop the world' GC problems that CSLM can cause. NM uses a read write lock, bulk reading KV pairs in the read lock to avoid contention. I think that part may not work for Cassandra which is more often iterative with it's reads than a purely MapReduce motivated BigTable design, however it could be improved on.

      1. https://github.com/apache/accumulo/blob/trunk/src/server/src/main/java/org/apache/accumulo/server/tabletserver/NativeMap.java

      Show
      Jason Rutherglen added a comment - It is probably worth looking at Accumolo's implementation of a NativeMap [1] that implements sorted key value pairs in C++ to avoid 'stop the world' GC problems that CSLM can cause. NM uses a read write lock, bulk reading KV pairs in the read lock to avoid contention. I think that part may not work for Cassandra which is more often iterative with it's reads than a purely MapReduce motivated BigTable design, however it could be improved on. 1. https://github.com/apache/accumulo/blob/trunk/src/server/src/main/java/org/apache/accumulo/server/tabletserver/NativeMap.java
      Hide
      Brandon Williams added a comment -

      'naive' patches (in that all I did was swap out CSLM) for lockfreeskiptree and snaptree. I didn't see any low hanging fruit improvement.

      Show
      Brandon Williams added a comment - 'naive' patches (in that all I did was swap out CSLM) for lockfreeskiptree and snaptree. I didn't see any low hanging fruit improvement.
      Hide
      Jason Rutherglen added a comment -

      Additionally, the idea would make use of Lucene's BytesRefHash [1,2] structure for primary key lookups (which is faster than CSLM's O(log N).

      1. http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/util/BytesRefHash.html
      2. http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene/src/java/org/apache/lucene/util/BytesRefHash.java

      Show
      Jason Rutherglen added a comment - Additionally, the idea would make use of Lucene's BytesRefHash [1,2] structure for primary key lookups (which is faster than CSLM's O(log N). 1. http://lucene.apache.org/java/3_4_0/api/all/org/apache/lucene/util/BytesRefHash.html 2. http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene/src/java/org/apache/lucene/util/BytesRefHash.java
      Hide
      Jason Rutherglen added a comment -

      For GC improvements, a strict array based approach could work well. I was going to test this in LUCENE-2312 for the terms dict. The idea is to merge sort arrays into larger structures. It's admittedly crude however in some tests it's very fast. Arrays can be easily pooled, further reducing garbage collection.

      Show
      Jason Rutherglen added a comment - For GC improvements, a strict array based approach could work well. I was going to test this in LUCENE-2312 for the terms dict. The idea is to merge sort arrays into larger structures. It's admittedly crude however in some tests it's very fast. Arrays can be easily pooled, further reducing garbage collection.

        People

        • Assignee:
          Brandon Williams
          Reporter:
          Jonathan Ellis
        • Votes:
          0 Vote for this issue
          Watchers:
          4 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development