[IGNITE-17177] Implement incremental snapshots

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.15
• Labels: IEP-89 important ise
• Release Note: Added new feature: Incremental snapshots
• Ignite Flags: Docs Required, Release Notes Required
• Assignee: Maksim Timonin
• Reporter: Maksim Timonin
• Reviewer: Nikolay Izhikov
• Created: 15/Jun/22 15:52
• Updated: 26/Apr/23 18:26
• Resolved: 09/Mar/23 20:10

Description

Solution is incremental snapshots based on the Consistent Cut algorithm and collecting WAL segments.

IEP-89: https://cwiki.apache.org/confluence/display/IGNITE/IEP-89%3A+Incremental+Snapshots

Issue Links

• relates to: IGNITE-17029 Implement Consistent Cut algorithm
• links to: GitHub Pull Request #10472
• links to: GitHub Pull Request #10539

Sub-Tasks

• Implement Consistent Cut algorithm: Resolved
• Add last WAL segment into SnapshotMetadata: Resolved
• Create incremental snapshot: Resolved
• Extract striped executor from memory restore to separate class: Resolved
• Restore incremental snapshot: Resolved
• [IEP-89] Incremental snapshot should restore binary_meta: Resolved
• Failed to find IncrementalSnapshotFinishRecord if snapshot has multiple segments: Resolved
• CacheStripedExecutor should handle error during execution of submitted task: Resolved
• Warn if Atomic cache is inconsistent after incremental snapshot restore: Resolved
• Joining clients should ignore incremental snapshot and their transaction must be excluded: Resolved
• Add system view for incremental snapshot: Resolved
• Add to SnapshotMXBean opportunity to restore incremental snapshot: Resolved
• Create metrics for incremental snapshot: Resolved
• Check incremental snapshot meta: Resolved
• Disable incremental snapshots creation after rebalance: Resolved
• Incremental snapshots dirs order should match full snapshot dirs: Resolved
• Add incIdx param to control.sh snapshot restore command: Resolved
• Add yardstick test for incremental snapshots: Resolved
• Add docs for incremental snapshots: Resolved