[ZOOKEEPER-1366] Zookeeper should be tolerant of clock adjustments

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.5.1, 3.6.0
• Assignee: Hongchao Deng
• Reporter: Ted Dunning
• Created: 19/Jan/12 07:00
• Updated: 11/May/17 20:34
• Resolved: 06/Feb/15 05:19

Description

If you want to wreak havoc on a ZK based system just do [date -s "+1hour"] and watch the mayhem as all sessions expire at once.

This shouldn't happen. Zookeeper could easily know handle elapsed times as elapsed times rather than as differences between absolute times. The absolute times are subject to adjustment when the clock is set while a timer is not subject to this problem. In Java, System.currentTimeMillis() gives you absolute time while System.nanoTime() gives you time based on a timer from an arbitrary epoch.

I have done this and have been running tests now for some tens of minutes with no failures. I will set up a test machine to redo the build again on Ubuntu and post a patch here for discussion.

Issue Links

• is duplicated by: ZOOKEEPER-366 Session timeout detection can go wrong if the leader system time changes
• is duplicated by: ZOOKEEPER-1616 time calculations should use a monotonic clock
• is required by: ZOOKEEPER-2774 Ephemeral znode will not be removed when sesstion timeout, if the system time of ZooKeeper node changes unexpectedly.
• relates to: ZOOKEEPER-1626 Zookeeper C client should be tolerant of clock adjustments

Sub-Tasks

• Zookeeper C client should be tolerant of clock adjustments: Resolved