Uploaded image for project: 'Apache Cassandra'
  1. Apache Cassandra
  2. CASSANDRA-6488

Batchlog writes consume unnecessarily large amounts of CPU on vnodes clusters

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 1.2.13, 2.0.4
    • None
    • None
    • Normal

    Description

      The cloneTokenOnlyMap call in StorageProxy.getBatchlogEndpoints causes enormous amounts of CPU to be consumed on clusters with many vnodes. I created a patch to cache this data as a workaround and deployed it to a production cluster with 15,000 tokens. CPU consumption drop to 1/5th. This highlights the overall issues with cloneOnlyTokenMap() calls on vnodes clusters. I'm including the maybe-not-the-best-quality workaround patch to use as a reference, but cloneOnlyTokenMap is a systemic issue and every place it's called should probably be investigated.

      Attachments

        1. 6488-fix.txt
          4 kB
          Aleksey Yeschenko
        2. 6488-rbranson-patch.txt
          5 kB
          Rick Branson
        3. 6488-v2.txt
          6 kB
          Jonathan Ellis
        4. 6488-v3.txt
          10 kB
          Aleksey Yeschenko
        5. graph (21).png
          41 kB
          Rick Branson

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            rbranson Rick Branson Assign to me
            rbranson Rick Branson
            Rick Branson
            Aleksey Yeschenko
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment