[CASSANDRA-4066] Cassandra cluster stops responding on time change (scheduling not using monotonic time?)

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Won't Fix
• Fix Version/s: 1.1.1
• Labels: gossip

Environment

Linux; CentOS6 2.6.32-220.4.2.el6.x86_64

• Severity: Low
• Assignee: Brandon Williams
• Reporter: David Daeschler

Authors

Brandon Williams

• Created: 20/Mar/12 15:45
• Updated: 16/Apr/19 09:32
• Resolved: 20/Mar/12 18:47

Description

The server installation I set up did not have ntpd installed in the base installation. When I noticed that the clocks were skewing I installed ntp and set the date on all the servers in the cluster. A short time later, I started getting UnavailableExceptions on the clients.

Also, one sever seemed to be unaffected by the time change. That server happened to have it's time pushed forward, not backwards like the other 3 in the cluster. This leads me to believe something is running on a timer/schedule that is not monotonic.

I'm posting this as a bug, but I suppose it might just be part of the communication protocols etc for the cluster and part of the design. But I think the devs should be aware of what I saw.

Otherwise, thank you for a fantastic product. Even after restarting 75% of the cluster things seem to have recovered nicely.