Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-7956

"nodetool compactionhistory" crashes because of low heap size (GC overhead limit exceeded)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Low
    • Resolution: Fixed
    • Fix Version/s: 2.0.11, 2.1.1
    • Component/s: None
    • Labels:
    • Environment:

      Cassandra 2.0.8

    • Severity:
      Low

      Description

      ]# nodetool compactionhistory
      Compaction History:
      Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
              at java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:967)
              at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1782)
              at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
              at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
              at java.util.HashMap.readObject(HashMap.java:1180)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
              at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
              at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
              at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
              at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
              at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
              at javax.management.openmbean.TabularDataSupport.readObject(TabularDataSupport.java:912)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
              at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
              at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
              at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
              at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
              at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:325)
              at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:174)
              at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
              at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
              at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:906)
              at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:267)
              at com.sun.proxy.$Proxy3.getCompactionHistory(Unknown Source)
      

      nodetool starts with -Xmx32m. This seems to be not enough at least in my case to show the history. I am not sure what would the appropriate amount be but increasing it to 128m definitely solves the problem. Output from modified nodetool attached.

        Attachments

        1. nodetool_compactionhistory_128m_heap_output.txt.gz
          1014 kB
          Nikolai Grigoriev
        2. 7956.txt
          0.4 kB
          Michael Shuler

          Issue Links

            Activity

              People

              • Assignee:
                mshuler Michael Shuler
                Reporter:
                ngrigoriev@gmail.com Nikolai Grigoriev
                Authors:
                Michael Shuler
                Reviewers:
                Jonathan Ellis
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: