[CASSANDRA-1310] Disallow KS definition with RF > # of nodes

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 0.7 beta 1

Environment

CentOS 5.1
Trunc July 22nd

• Severity: Normal
• Assignee: Nate McCall
• Reporter: Arya Goudarzi

Authors

Nate McCall

• Created: 23/Jul/10 01:15
• Updated: 16/Apr/19 09:33
• Resolved: 29/Jul/10 15:59

Description

Cassandra 0.7 allows user to create Keyspaces with Replication Factor > number of endpoints causing in java.lang.IllegalStateException: replication factor (2) exceeds number of endpoints (1) exception in nodetool and Internal Errors on Thrift making the node useless.

Steps to Reproduce:

From a clean setup of Cassandra:
1. Start a single node out of cluster of 3. This means my configuration has the other two nodes in the seeds list, but have not restarted them yet;
2. Use Thrift API (I am using PHP) and create a Keyspace with replication factor 2;
3. The command executes with no exception or error;
4. Now try writing to it, you will get TException with Internal Error message;
5. Try nodetool ring and you will get Exception:

Exception in thread "main" java.lang.IllegalStateException: replication factor (2) exceeds number of endpoints (1)
at org.apache.cassandra.locator.RackUnawareStrategy.calculateNaturalEndpoints(RackUnawareStrategy.java:61)
at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:87)
at org.apache.cassandra.service.StorageService.constructRangeToEndpointMap(StorageService.java:536)
at org.apache.cassandra.service.StorageService.getRangeToAddressMap(StorageService.java:522)
at org.apache.cassandra.service.StorageService.getRangeToEndpointMap(StorageService.java:496)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226)
at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:251)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:857)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:795)
at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1449)
at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:90)
at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1284)
at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1382)
at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:807)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
at sun.rmi.transport.Transport$1.run(Transport.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)

Expected:

1. Either step 3 should not let you create the KS with RF 2 and 1 node in ring, or there should be a peaceful way for Cassandra to recover from IllegalStateException and replicate once other nodes become available.