Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
None
-
Linux 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686 GNU/Linux, ec2 small instance
-
Normal
Description
The following exception occurs consistently on at least node (note did not occur on other same-configured nodes) during startup:
INFO - Replaying /var/lib/cassandra/commitlog/CommitLog-1262855754427.log, /var/lib/cassandra/commitlog/CommitLog-1262832689989.log, /var/lib/cassandra/commitlog/CommitLog-1262885833186.log, /var/lib/cassandra/commitlog/CommitLog-1262900845019.log, /var/lib/cassandra/commitlog/CommitLog-1262913267844.log, /var/lib/cassandra/commitlog/CommitLog-1262927898170.log, /var/lib/cassandra/commitlog/CommitLog-1262961421039.log, /var/lib/cassandra/commitlog/CommitLog-1262977175175.log, /var/lib/cassandra/commitlog/CommitLog-1262989588783.log, /var/lib/cassandra/commitlog/CommitLog-1263000573676.log, /var/lib/cassandra/commitlog/CommitLog-1263013691393.log, /var/lib/cassandra/commitlog/CommitLog-1263044706108.log, /var/lib/cassandra/commitlog/CommitLog-1263060004191.log, /var/lib/cassandra/commitlog/CommitLog-1263071446342.log, /var/lib/cassandra/commitlog/CommitLog-1263082950154.log, /var/lib/cassandra/commitlog/CommitLog-1263095400814.log, /var/lib/cassandra/commitlog/CommitLog-1263118331046.log, /var/lib/cassandra/commitlog/CommitLog-1263143402963.log, /var/lib/cassandra/commitlog/CommitLog-1263155294308.log, /var/lib/cassandra/commitlog/CommitLog-1263166154352.log, /var/lib/cassandra/commitlog/CommitLog-1263178359247.log, /var/lib/cassandra/commitlog/CommitLog-1263202112017.log, /var/lib/cassandra/commitlog/CommitLog-1263230932274.log, /var/lib/cassandra/commitlog/CommitLog-1263250726505.log, /var/lib/cassandra/commitlog/CommitLog-1263264159438.log, /var/lib/cassandra/commitlog/CommitLog-1263289964249.log, /var/lib/cassandra/commitlog/CommitLog-1263317974387.log, /var/lib/cassandra/commitlog/CommitLog-1263331989090.log, /var/lib/cassandra/commitlog/CommitLog-1263344147667.log, /var/lib/cassandra/commitlog/CommitLog-1263359751527.log, /var/lib/cassandra/commitlog/CommitLog-1263395707008.log, /var/lib/cassandra/commitlog/CommitLog-1263397833524.log, /var/lib/cassandra/commitlog/CommitLog-1263398736183.log, /var/lib/cassandra/commitlog/CommitLog-1263399753707.log, /var/lib/cassandra/commitlog/CommitLog-1263401667504.log, /var/lib/cassandra/commitlog/CommitLog-1263404640782.log, /var/lib/cassandra/commitlog/CommitLog-1263405827234.log, /var/lib/cassandra/commitlog/CommitLog-1263406901115.log
INFO - LocationInfo has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(LocationInfo)@25934689
INFO - HintsColumnFamily has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(HintsColumnFamily)@4766820
INFO - AdXRequestStatistics has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(AdXRequestStatistics)@21521158
INFO - TokenGoogleIDCF has reached its threshold; switching in a fresh Memtable
INFO - Enqueuing flush of Memtable(TokenGoogleIDCF)@22889075
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.commons.daemon.support.DaemonLoader.load(DaemonLoader.java:160)
Caused by: java.lang.AssertionError: Blocking serialized executor is not yet implemented
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
at org.apache.cassandra.db.Table.flush(Table.java:464)
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:397)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:65)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:90)
at org.apache.cassandra.service.CassandraDaemon.init(CassandraDaemon.java:135)
... 5 more
And the same exception occurs intermittently on other node (running) nodes during 'nodeprobe flush':
root@domU-12-31-38-00-26-31:~# nodeprobe -host localhost -port 8080 flush Logger
Exception in thread "main" java.lang.AssertionError: Blocking serialized executor is not yet implemented
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:84)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:78)
at org.apache.cassandra.db.ColumnFamilyStore.submitFlush(ColumnFamilyStore.java:1045)
at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:395)
at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:448)
at org.apache.cassandra.service.StorageService.forceTableFlush(StorageService.java:984)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:93)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:27)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:208)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:120)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:262)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:836)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:761)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1426)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:72)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1264)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1359)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:788)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
at sun.rmi.transport.Transport$1.run(Transport.java:159)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)