Cassandra
  1. Cassandra
  2. CASSANDRA-1878

Minimize Key Cache Invalidation by Compaction

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 0.6.9, 0.7.0 rc 3
    • Component/s: Core
    • Labels:
      None

      Description

      Currently, compactions invalidate key cache entries that pointed to the SSTables that were compacted. This results in a sudden increase in the number of seeks necessary for reads immediately after compaction. When writing out a new SSTable after a compaction, it seems like it should be possible to store a list of keys that are currently cached but would be invalidated along with their new position in the new SSTable. Matt Dennis also seems to think this would be relatively easy to do.

      1. 1878.txt
        4 kB
        Jonathan Ellis
      2. 1878_v2.txt
        3 kB
        T Jake Luciani

        Activity

        Tyler Hobbs created issue -
        Hide
        Jonathan Ellis added a comment -

        patch to re-cache hot keys post-compaction

        Show
        Jonathan Ellis added a comment - patch to re-cache hot keys post-compaction
        Jonathan Ellis made changes -
        Field Original Value New Value
        Attachment 1878.txt [ 12466538 ]
        Hide
        Jonathan Ellis added a comment -

        (patch is against 0.6)

        Show
        Jonathan Ellis added a comment - (patch is against 0.6)
        Jonathan Ellis made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Jonathan Ellis [ jbellis ]
        Reviewer tjake
        Fix Version/s 0.6.9 [ 12315556 ]
        Fix Version/s 0.7.0 [ 12315950 ]
        Hide
        T Jake Luciani added a comment -

        Tested before and after compaction with and without patch, and saw the KeyCache hit rate remain high with this patch post compaction. Also, the Key Cache hit rate is no longer NaN after compaction.

        I removed a mistaken include in v2.

        +1

        Show
        T Jake Luciani added a comment - Tested before and after compaction with and without patch, and saw the KeyCache hit rate remain high with this patch post compaction. Also, the Key Cache hit rate is no longer NaN after compaction. I removed a mistaken include in v2. +1
        T Jake Luciani made changes -
        Attachment 1878_v2.txt [ 12466583 ]
        Hide
        Jonathan Ellis added a comment -

        committed

        Show
        Jonathan Ellis added a comment - committed
        Jonathan Ellis made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Cassandra-0.6 #29 (See https://hudson.apache.org/hudson/job/Cassandra-0.6/29/)
        Re-cache hotkeys post-compaction without hitting disk
        patch by jbellis; reviewed by tjake for CASSANDRA-1878

        Show
        Hudson added a comment - Integrated in Cassandra-0.6 #29 (See https://hudson.apache.org/hudson/job/Cassandra-0.6/29/ ) Re-cache hotkeys post-compaction without hitting disk patch by jbellis; reviewed by tjake for CASSANDRA-1878
        Gavin made changes -
        Workflow no-reopen-closed, patch-avail [ 12540471 ] patch-available, re-open possible [ 12752553 ]
        Gavin made changes -
        Workflow patch-available, re-open possible [ 12752553 ] reopen-resolved, no closed status, patch-avail, testing [ 12755421 ]

          People

          • Assignee:
            Jonathan Ellis
            Reporter:
            Tyler Hobbs
            Reviewer:
            T Jake Luciani
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development