[CASSANDRA-1746] Cleanups should be less impacting

• Type: New Feature
• Status: Resolved
• Priority: Low
• Resolution: Won't Fix
• Assignee: Unassigned
• Reporter: Edward Capriolo
• Created: 15/Nov/10 17:14
• Updated: 16/Apr/19 09:33
• Resolved: 30/Apr/11 16:51

Description

When a new node is added its neighbours require cleanup. Cleanup is very performance impacting and for larger data sets takes a long time. You really do not get all the benefits of the new node until the neighbours are cleaned up.
Suggestion:
Configuration option that can be changed from JMX compaction_auto_cleanup :=

{true,false}

set to false by default.
During non major compaction if compaction_auto_cleanup flag is set to TRUE, we look at the natural endpoints for the key we are compacting. If the key does not belong on this machine we can remove it.

This would save us from the heavy hammer of cleanup compaction. It would also be less book keeping for administrators.
Most people would want to leave this at false, join new node, wait a few days. If the node has not failed by now, it likely will not. Set the flag to true and cleanup will happen over time. Users can still force clean up if they wish.