[IGNITE-17254] Storage API for RAFT snapshot streaming

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-beta1
• Labels: ignite-3
• Epic Link: Storage support for Rebalancing
• Assignee: Roman Puchkovskiy
• Reporter: Roman Puchkovskiy
• Reviewer: Semyon Danilov
• Created: 28/Jun/22 18:22
• Updated: 31/Oct/22 13:49
• Resolved: 13/Oct/22 13:54

Description

Current PartitionStorage has a method for taking snapshots (and a method for restoring from them). The  methods work with files. In IGNITE-17253, we are going to switch to streaming protocol (which does not use files at all). So we need streaming snapshot-related methods on MvPartitionStorage.

UPDATE

From my understanding, we basically need two more methods in the MV partition storage to manipulate data:

// This one represents a combination of "addWrite" and "commitWrite", but a faster one.
// It will be used in incoming snapshot copier to write data.
void addWriteCommitted(RowId rowId, BinaryRow row, HybridTimestamp commitTimestamp);

// This one will be used to get all versions of a particular row and form a SnapshotMvDataResponse message.
Cursor<ReadResult> scanVersionsEx(RowId rowId);

Both of them are very simple and, for the most part, match their counterparts in every storage.

What's not as simple is the fail-over scenario. We shouldn't treat partially-downloaded partition as a valid set of data. So, the easiest was (in My opinion) to achieve this is to:

• set "lastAppliedIndex" value to "-1" before writing any the data
• drop old data (these two steps may be swapped, it's not that important)
• download and write everything
• set valid value for "lastAppliedIndex"

This is a crude solution, but should work well, especially if properly documented.

Issue Links

• fixes: IGNITE-17265 Support RAFT snapshot streaming for PageMemory storage
• split from: IGNITE-17083 Base classes for full rebalance procedure
• links to: GitHub Pull Request #1194