Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-12462

NullPointerException in CompactionInfo.getId(CompactionInfo.java:65)

    XMLWordPrintableJSON

    Details

    • Severity:
      Normal

      Description

      Note: The same trace is cited in the last comment of https://issues.apache.org/jira/browse/CASSANDRA-11961

      I've noticed that some of my nodes in my 2.1 cluster have fallen way behind on compactions, and have huge numbers (thousands) of uncompacted, tiny SSTables (~30MB or so).

      In diagnosing the issue, I've found that "nodetool compactionstats" returns the exception below. Restarting cassandra on the node here causes the pending tasks count to jump to ~2000. Compactions run properly for about an hour, until this exception occurs again. Once it occurs, I see the pending tasks value rapidly drop towards zero, but without any compactions actually running (the logs show no compactions finishing). It would seem that this is causing compactions to fail on this node, which is leading to it running out of space, etc.

      [redacted]# nodetool compactionstats
      xss = -ea -javaagent:/usr/share/cassandra/lib/jamm-0.3.0.jar -XX:+UseThreadPriorities -XX:ThreadPriorityPolicy=42 -Xms12G -Xmx12G -Xmn1000M -Xss255k
      pending tasks: 5
      error: null
      – StackTrace –
      java.lang.NullPointerException
      at org.apache.cassandra.db.compaction.CompactionInfo.getId(CompactionInfo.java:65)
      at org.apache.cassandra.db.compaction.CompactionInfo.asMap(CompactionInfo.java:118)
      at org.apache.cassandra.db.compaction.CompactionManager.getCompactions(CompactionManager.java:1405)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at sun.reflect.misc.Trampoline.invoke(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
      at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
      at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
      at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
      at com.sun.jmx.mbeanserver.PerInterface.getAttribute(Unknown Source)
      at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(Unknown Source)
      at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(Unknown Source)
      at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(Unknown Source)
      at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
      at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown Source)
      at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
      at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
      at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(Unknown Source)
      at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown Source)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      at java.lang.reflect.Method.invoke(Unknown Source)
      at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
      at sun.rmi.transport.Transport$1.run(Unknown Source)
      at sun.rmi.transport.Transport$1.run(Unknown Source)
      at java.security.AccessController.doPrivileged(Native Method)
      at sun.rmi.transport.Transport.serviceCall(Unknown Source)
      at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
      at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
      at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      at java.lang.Thread.run(Unknown Source)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                szhou Simon Zhou
                Reporter:
                jdeprizio Jonathan DePrizio
                Authors:
                Simon Zhou
                Reviewers:
                Yuki Morishita
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: