[CASSANDRA-14190] Non-disruptive seed node list reload

• Type: Improvement
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 4.0-alpha1, 4.0
• Component/s: Local/Config, Local/Startup and Shutdown
• Assignee: Samuel Fink
• Reporter: Samuel Fink

Authors

Samuel Fink

Reviewers

Jason Brown

• Created: 24/Jan/18 18:21
• Updated: 15/May/20 08:03
• Resolved: 31/Jan/18 00:33

Description

Add a mechanism for reloading the Gossiper in-memory seed node IP list without requiring a service restart.

The Gossiper keeps an in-memory copy of the seed node IP list and uses it during a gossip round to determine if the random node that was gossiped to is a seed node and for picking a seed node to gossip to in maybeGossipToSeed.

Currently the Gossiper seed node list is only updated when an endpoint is removed, at the start of a shadow round, and on startup. Those scenarios don’t handle the case of seed nodes changing IP addresses (eg. DHCP lease changes) or additional seed nodes being added to the cluster.

As described in CASSANDRA-3829 the current way to ensure that all nodes in the cluster have the same seed node list when there has been a change is to do a rolling restart of every node in the cluster. In large clusters rolling restarts can be very complicated to manage and can have performance impacts because the caches get flushed.