Details
Major Description
Incremental snapshot is a lightweight alternative to full snapshot. It bases on the non-blocking Consistent Cut algorithm and provides a collection of WAL segments that hold logical changes since previous snapshot (full or incremental).
Incremental snapshot should contain:
- compacted WAL segments
- meta file with Consistent Cut to restore on
- binary_meta if it has changed since previous snapshot.
Incremental snapshot is stored within full snapshot directory.
Incremental snapshot before creation checks:
- Base snapshot (at least its metafile) exists. Exists metafile for all incremental snapshots.
- Validate that no misses in WAL segments since previous snapshot.
- Check that new ConsistentCutVersion is greater than version of previous snapshots.
- Check that baseline topology and cacheGroups are the same (relatively to base snapshot).
More info in IEP: https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=211884314
Highlevel process of creation of incremental snapshot(without consistent cut algorithm):
- Incremental snapshot started with create snapshot command with --incremental flag.
- Incremental snapshot consists of:
- WAL segments from previous increment snapshot (or full snapshot in case first incremental snapshot).
- Changed binary meta and marshaller files.
- Snapshot metafile
- Incremental snapshot are placed to work/snapshots/mybackup/increments/node01/0001 folder where
- mybackup - name of the full snapshot.
- node01 - consistent id of the node.
- 0001, 0002, etc - number of incremental snapshot.
- Incremental snapshot creation consists of the following actions executed on each node. The whole process orchestrated by the DistributedProcess in the same manner as the full snapshot creation:
- creation of the snapshot folder
- awaits while required WAL segments will be archived
- copy (hard-linked) required WAL segments to the incremental snapshot folder.
- creates snapshot metafile.
- Failover guarantees (remove partially created snapshot, etc) should be the same as for the full snapshot.
- Removal of the full snapshot must also removes all the incremental snapshot based on the full one.
- Removal of the incremental snapshot may be done only for the last one. If there are next incremental snapshot (0003, for example) then removal of any previous (0001 or 0002) must be restricted.
