[ZOOKEEPER-3115] Delete snapshot file on error

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.6.0
• Component/s: server
• Assignee: kevin.chen
• Reporter: Brian Nixon
• Created: 09/Aug/18 01:12
• Updated: 20/Aug/18 09:12

Description

ZOOKEEPER-3082 guards against one particular failure mode that can cause a corrupt snapshot, when a empty file is created with a valid snapshot file name. All other instances of IOException when writing the snapshot are simply allowed to propagate up the stack.

One idea that came up during review (https://github.com/apache/zookeeper/pull/560) was whether we would ever want to leave a snapshot file on disk when an IOException is thrown. Clearly something has gone wrong at this point and rather than leave a potentially corrupt file, we can delete it and trust the transaction log when restoring the necessary transactions.

It would be great to modify FileTxnSnapLog::save to delete snapshot files more often on exceptions - provided that there's a way to identify when the file in that case is needed or corrupt.