[ZOOKEEPER-3419] Backup and recovery support

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.6.0
• Component/s: server
• Labels: Twitter pull-request-available
• Assignee: Michael Han
• Reporter: Michael Han
• Created: 06/Jun/19 22:31
• Updated: 13/Jun/22 14:30

Description

Historically ZooKeeper has no intrinsic support for backup and restore. The usual approach of doing backup and restore is through customized scripts to copy data around, or through some 3rd party tools (exhibitor, etc), which introduces operation burden. 

This Jira will introduce another option: a direct support of backup and restore from ZooKeeper itself. It's completely built into ZooKeeper, support point in time recovery of an entire tree rooted after an oops event, support recovery partial tree for test/dev purpose, and can help replay history for bug investigation. It will try to provide a generic interface so the backups can be directed to different data storage systems (S3, Kafka, HDFS, etc).

This same system has been in production at Twitter for X years and proved to be quite helpful for various use cases mentioned earlier. This will be a relative big patch, we'll try break the feature down and incrementally submit the patches when they are ready.

Issue Links

• links to: GitHub Pull Request #1492