Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-8716

"java.util.concurrent.ExecutionException: java.lang.AssertionError: Memory was freed" when running cleanup

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Low
    • Resolution: Fixed
    • Fix Version/s: 2.0.13
    • Component/s: None
    • Labels:
    • Environment:

      Centos 6.6, Cassandra 2.0.12, Oracle JDK 1.7.0_67

    • Severity:
      Low

      Description

      Error occurred during cleanup
      java.util.concurrent.ExecutionException: java.lang.AssertionError: Memory was freed
              at java.util.concurrent.FutureTask.report(FutureTask.java:122)
              at java.util.concurrent.FutureTask.get(FutureTask.java:188)
              at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTableOperation(CompactionManager.java:234)
              at org.apache.cassandra.db.compaction.CompactionManager.performCleanup(CompactionManager.java:272)
              at org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:1115)
              at org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(StorageService.java:2177)
              at sun.reflect.GeneratedMethodAccessor29.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:75)
              at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:279)
              at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
              at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
              at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
              at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
              at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
              at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
              at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
              at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1487)
              at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:97)
              at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1328)
              at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1420)
              at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:848)
              at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
              at sun.rmi.transport.Transport$1.run(Transport.java:177)
              at sun.rmi.transport.Transport$1.run(Transport.java:174)
              at java.security.AccessController.doPrivileged(Native Method)
              at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
              at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:556)
              at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:811)
              at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:670)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.AssertionError: Memory was freed
              at org.apache.cassandra.io.util.Memory.checkPosition(Memory.java:259)
              at org.apache.cassandra.io.util.Memory.getInt(Memory.java:211)
              at org.apache.cassandra.io.sstable.IndexSummary.getIndex(IndexSummary.java:79)
              at org.apache.cassandra.io.sstable.IndexSummary.getKey(IndexSummary.java:84)
              at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:58)
              at org.apache.cassandra.io.sstable.SSTableReader.getIndexScanPosition(SSTableReader.java:602)
              at org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:947)
              at org.apache.cassandra.io.sstable.SSTableReader.getPosition(SSTableReader.java:910)
              at org.apache.cassandra.io.sstable.SSTableReader.getPositionsForRanges(SSTableReader.java:819)
              at org.apache.cassandra.db.ColumnFamilyStore.getExpectedCompactedFileSize(ColumnFamilyStore.java:1088)
              at org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:564)
              at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:63)
              at org.apache.cassandra.db.compaction.CompactionManager$5.perform(CompactionManager.java:281)
              at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:225)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              ... 3 more

      The error continue to happen even after scrub.
      If I retry a few time, it may run successfully.
      I am attaching the server log.

        Attachments

        1. system.log.gz
          165 kB
          Imri Zvik
        2. 8716.txt
          2 kB
          Robert Stupp

          Issue Links

            Activity

              People

              • Assignee:
                snazy Robert Stupp
                Reporter:
                imriz Imri Zvik
                Authors:
                Robert Stupp
                Reviewers:
                Benedict
                Tester:
                Philip Thompson
              • Votes:
                1 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: