[ZOOKEEPER-3419] Backup and recovery support - ASF JIRA

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.6.0
Fix Version/s: None
Component/s: server
Labels:
- Twitter
- pull-request-available

Description

Historically ZooKeeper has no intrinsic support for backup and restore. The usual approach of doing backup and restore is through customized scripts to copy data around, or through some 3rd party tools (exhibitor, etc), which introduces operation burden.

This Jira will introduce another option: a direct support of backup and restore from ZooKeeper itself. It's completely built into ZooKeeper, support point in time recovery of an entire tree rooted after an oops event, support recovery partial tree for test/dev purpose, and can help replay history for bug investigation. It will try to provide a generic interface so the backups can be directed to different data storage systems (S3, Kafka, HDFS, etc).

This same system has been in production at Twitter for X years and proved to be quite helpful for various use cases mentioned earlier. This will be a relative big patch, we'll try break the feature down and incrementally submit the patches when they are ready.