Uploaded image for project: 'Apache Cassandra'
  1. Apache Cassandra
  2. CASSANDRA-5554

Growing pending compactions

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Normal
    • Resolution: Fixed
    • 1.2.5
    • None
    • None
    • Normal

    Description

      I noticed on one of our new cassandra production server, that "pending compactions" number is steadily growing. The cluster is under low write load, so "compactions are not keeping up" was not the case.

      A quick investigation shown, that compactions are stopping far before all pending tasks are completed. I also found, that if concurrent_compactors=1, background compactions are not happening at all.

      The bug is in BackgroundCompactionTask rescheduling logic. The executor pool "room control" code in CompactionManager.submitBackground() does not reschedule next background cycle, if executor.getActiveCount reach maximun pool size, so it is lost forever.

      So I patched it to always schedule single background cycle, regardless of the free room in executor pool.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            m0nstermind Oleg Anastasyev Assign to me
            m0nstermind Oleg Anastasyev
            Oleg Anastasyev
            Jonathan Ellis
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment