[CASSANDRA-9652] Nodetool cleanup does not work for nodes taken out of replication

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 3.11.5
• Severity: Low
• Since Version: 2.0.10
• Assignee: Unassigned
• Reporter: Erick Ramirez (deprecated)
• Created: 25/Jun/15 13:39
• Updated: 01/Aug/21 11:28
• Resolved: 04/Jul/18 01:31

Description

After taking a node (DC) out of replication, running a cleanup does not get rid of the data on the node. The SSTables remain on disk and no data is cleared out.

The following entry is recorded in system.log:

 INFO [CompactionExecutor:8] 2015-06-25 12:33:01,417 CompactionManager.java (line 527) Cleanup cannot run before a node has joined the ring

STEPS TO REPRODUCE

1. Build a (C* 2.0.10) cluster with multiple DCs.
2. Run cassandra-stress -n1 to create schema.
3. Alter schema to replicate to all DCs.

   cqlsh> ALTER KEYSPACE "Keyspace1" WITH replication = { 'class' : 'NetworkTopologyStrategy', 'DC1' : 2, 'DC2' : 2, 'DC3' : 1 } ;
   

4. Run cassandra-stress -n100000 to generate data.
5. Alter schema to stop replication to DC3.
6. On node in DC3, run nodetool cleanup.

WORKAROUND

1. Stop Cassandra.
2. Manually delete the SSTables on disk.
3. Start Cassandra.

Issue Links

• is superceded by: CASSANDRA-13526 nodetool cleanup on KS with no replicas should remove old data, not silently complete