[CASSANDRA-7956] "nodetool compactionhistory" crashes because of low heap size (GC overhead limit exceeded)

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.0.11, 2.1.1
• Labels: qa-resolved

Environment

Cassandra 2.0.8

• Severity: Low
• Assignee: Michael Shuler
• Reporter: Nikolai Grigoriev

Authors

Michael Shuler

Reviewers

Jonathan Ellis

• Created: 17/Sep/14 14:18
• Updated: 16/Apr/19 09:31
• Resolved: 01/Oct/14 17:38

Description

]# nodetool compactionhistory
Compaction History:
Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:967)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1782)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
        at java.util.HashMap.readObject(HashMap.java:1180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
        at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:500)
        at javax.management.openmbean.TabularDataSupport.readObject(TabularDataSupport.java:912)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1893)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1798)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
        at sun.rmi.server.UnicastRef.unmarshalValue(UnicastRef.java:325)
        at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:174)
        at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source)
        at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source)
        at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:906)
        at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:267)
        at com.sun.proxy.$Proxy3.getCompactionHistory(Unknown Source)

nodetool starts with -Xmx32m. This seems to be not enough at least in my case to show the history. I am not sure what would the appropriate amount be but increasing it to 128m definitely solves the problem. Output from modified nodetool attached.

Issue Links

• is related to: CASSANDRA-7238 Nodetool Status performance is much slower with VNodes On