Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-8210

"java.lang.AssertionError: Memory was freed" exception in CompactionExecutor

Agile BoardAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Low
    • Resolution: Duplicate
    • None
    • None
    • None
    • DSE 4.5.2, Cassandra 2.0.10, OEL 6.5, kernel 3.8.13-44.el6uek.x86_64, 128Gb of RAM, swap disabled, JRE 1.7.0_67-b01

    • Low

    Description

      I have just got this problem on multiple nodes. Cassandra 2.0.10 (DSE 4.5.2). After looking through the history I have found that it was actually happening on all nodes since the start of large compaction process (I've loaded tons of data in the system and then turned off all load to let it compact the data).

      ERROR [CompactionExecutor:1196] 2014-10-28 17:14:50,124 CassandraDaemon.java (line 199) Exception in thread Thread[CompactionExecutor:1196,1,main]
      java.lang.AssertionError: Memory was freed
              at org.apache.cassandra.io.util.Memory.checkPosition(Memory.java:259)
              at org.apache.cassandra.io.util.Memory.getInt(Memory.java:211)
              at org.apache.cassandra.io.sstable.IndexSummary.getIndex(IndexSummary.java:79)
              at org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:84)
              at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:58)
              at org.apache.cassandra.io.sstable.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:692)
              at org.apache.cassandra.io.sstable.SSTableReader.estimatedKeysForRanges(SSTableReader.java:663)
              at org.apache.cassandra.db.compaction.AbstractCompactionStrategy.worthDroppingTombstones(AbstractCompactionStrategy.java:328)
              at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.findDroppableSSTable(LeveledCompactionStrategy.java:354)
              at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getMaximalTask(LeveledCompactionStrategy.java:125)
              at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getNextBackgroundTask(LeveledCompactionStrategy.java:113)
              at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:192)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      

      Attachments

        1. system.log.gz
          1.15 MB
          Nikolai Grigoriev
        2. cassandra.yaml
          32 kB
          Nikolai Grigoriev
        3. cassandra-env.sh
          11 kB
          Nikolai Grigoriev
        4. occurence frequency.png
          138 kB
          Nikolai Grigoriev

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned Assign to me
            ngrigoriev@gmail.com Nikolai Grigoriev
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment