Cassandra
  1. Cassandra
  2. CASSANDRA-1878

Minimize Key Cache Invalidation by Compaction

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 0.6.9, 0.7.0 rc 3
    • Component/s: Core
    • Labels:
      None

      Description

      Currently, compactions invalidate key cache entries that pointed to the SSTables that were compacted. This results in a sudden increase in the number of seeks necessary for reads immediately after compaction. When writing out a new SSTable after a compaction, it seems like it should be possible to store a list of keys that are currently cached but would be invalidated along with their new position in the new SSTable. Matt Dennis also seems to think this would be relatively easy to do.

      1. 1878_v2.txt
        3 kB
        T Jake Luciani
      2. 1878.txt
        4 kB
        Jonathan Ellis

        Activity

        Hide
        Jonathan Ellis added a comment -

        patch to re-cache hot keys post-compaction

        Show
        Jonathan Ellis added a comment - patch to re-cache hot keys post-compaction
        Hide
        Jonathan Ellis added a comment -

        (patch is against 0.6)

        Show
        Jonathan Ellis added a comment - (patch is against 0.6)
        Hide
        T Jake Luciani added a comment -

        Tested before and after compaction with and without patch, and saw the KeyCache hit rate remain high with this patch post compaction. Also, the Key Cache hit rate is no longer NaN after compaction.

        I removed a mistaken include in v2.

        +1

        Show
        T Jake Luciani added a comment - Tested before and after compaction with and without patch, and saw the KeyCache hit rate remain high with this patch post compaction. Also, the Key Cache hit rate is no longer NaN after compaction. I removed a mistaken include in v2. +1
        Hide
        Jonathan Ellis added a comment -

        committed

        Show
        Jonathan Ellis added a comment - committed
        Hide
        Hudson added a comment -

        Integrated in Cassandra-0.6 #29 (See https://hudson.apache.org/hudson/job/Cassandra-0.6/29/)
        Re-cache hotkeys post-compaction without hitting disk
        patch by jbellis; reviewed by tjake for CASSANDRA-1878

        Show
        Hudson added a comment - Integrated in Cassandra-0.6 #29 (See https://hudson.apache.org/hudson/job/Cassandra-0.6/29/ ) Re-cache hotkeys post-compaction without hitting disk patch by jbellis; reviewed by tjake for CASSANDRA-1878

          People

          • Assignee:
            Jonathan Ellis
            Reporter:
            Tyler Hobbs
            Reviewer:
            T Jake Luciani
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development