When loading state from snapshots on startup, FileTxnSnapLog.java ignores the result of FileSnap.deserialize, which is -1L if no valid snapshots are found. Recovery proceeds with dt.lastProcessed == 0, its initial value.
The result is that Zookeeper will process the transaction logs and then begin serving requests with a different state than the rest of the ensemble.
In a healthy zookeeper cluster of size >= 3, shut down one node.
Either delete all snapshots for this node or change all to be empty files.
Restart the node.
We believe this can happen organically if a node runs out of disk space.