Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10274

Assertion Errors when interrupting Cleanup

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Urgent
    • Resolution: Fixed
    • Fix Version/s: 2.1.10, 2.2.2, 3.0.0 rc1
    • Component/s: None
    • Labels:
      None
    • Severity:
      Critical

      Description

      Exceptions encountered after Interrupting cleanup on 2.1.7 - logging due to nature. Seen on 2 different nodes. May be related to CASSANDRA-10260 .

      INFO  [CompactionExecutor:5836] 2015-09-06 11:33:39,630 CompactionManager.java:1286 - Compaction interrupted: Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, 44698243387/139158
      624314)bytes
      INFO  [CompactionExecutor:5838] 2015-09-06 11:33:39,638 CompactionManager.java:1286 - Compaction interrupted: Cleanup@74bffc10-0fbe-11e5-a5ce-37a8b36fe285(keyspace, table, 37886026781/133123
      638379)bytes
      INFO  [CompactionExecutor:5836] 2015-09-06 11:33:39,639 CompactionManager.java:749 - Cleaning up SSTableReader(path='/mnt/cassandra/data/keyspace/table-74bffc100fbe11e5a5ce37a8b36fe285/keyspace-table-26598-Data.db')
      ERROR [CompactionExecutor:5838] 2015-09-06 11:33:39,639 CassandraDaemon.java:223 - Exception in thread Thread[CompactionExecutor:5838,1,main]
      java.lang.AssertionError: Memory was freed
              at org.apache.cassandra.io.util.SafeMemory.checkBounds(SafeMemory.java:97) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.util.Memory.getInt(Memory.java:281) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummary.getPositionInSummary(IndexSummary.java:139) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:144) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:113) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:1183) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.SSTableReader.firstKeyBeyond(SSTableReader.java:1667) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.db.compaction.CompactionManager.needsCleanup(CompactionManager.java:693) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.db.compaction.CompactionManager.doCleanupOne(CompactionManager.java:734) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:94) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.db.compaction.CompactionManager$5.execute(CompactionManager.java:389) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:285) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_51]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_51]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
      
      

      And:

      ERROR [IndexSummaryManager:1] 2015-09-06 09:32:23,346 CassandraDaemon.java:223 - Exception in thread Thread[IndexSummaryManager:1,1,main]
      java.lang.AssertionError: null
              at org.apache.cassandra.io.sstable.SSTableReader.setReplacedBy(SSTableReader.java:955) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.SSTableReader.cloneAndReplace(SSTableReader.java:1002) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.SSTableReader.cloneWithNewSummarySamplingLevel(SSTableReader.java:1105) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummaryManager.adjustSamplingLevels(IndexSummaryManager.java:421) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:299) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummaryManager.redistributeSummaries(IndexSummaryManager.java:238) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.io.sstable.IndexSummaryManager$1.runMayThrow(IndexSummaryManager.java:139) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor$UncomplainingRunnable.run(DebuggableScheduledThreadPoolExecutor.java:118) ~[apache-cassandra-2.1.7.jar:2.1.7]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_51]
              at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_51]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_51]
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_51]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_51]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_51]
              at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51]
      ERROR [Reference-Reaper:1] 2015-09-06 09:32:24,891 Ref.java:179 - LEAK DETECTED: a reference (org.apache.cassandra.utils.concurrent.Ref$State@56b339db) to class org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@1484053885:/mnt/cassandra/data/keyspace/table-74bffc100fbe11e5a5ce37a8b36fe285/keyspace-table-ka-6021 was not released before the reference was garbage collected
      

        Attachments

          Activity

            People

            • Assignee:
              benedict Benedict Elliott Smith
              Reporter:
              jjirsa Jeff Jirsa
              Authors:
              Benedict Elliott Smith
              Reviewers:
              Marcus Eriksson
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: