Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-8312

Use live sstables in snapshot repair if possible

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Low
    • Resolution: Won't Do
    • Fix Version/s: 2.1.x
    • Component/s: None
    • Labels:

      Description

      Snapshot repair can be very much slower than parallel repairs because of the overhead of opening the SSTables in the snapshot. This is particular true when using LCS, as you typically have many smaller SSTables then.

      I compared parallel and sequential repair on a small range on one of our clusters (2*3 replicas). With parallel repair, this took 22 seconds. With sequential repair (default in 2.0), the same range took 330 seconds! This is an overhead of 330-22*6 = 198 seconds, just opening SSTables (there were 1000+ sstables). Also, opening 1000 sstables for many smaller rangers surely causes lots of memory churning.

      The idea would be to list the sstables in the snapshot, but use the corresponding sstables in the live set if it's still available. For almost all sstables, the original one should still exist.

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              Unassigned Assign to me
              Reporter:
              yarin Jimmy Mårdell

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment