[IGNITE-17232] Optimization of DeltaFilePageStore: write new pages directly to FilePageStore

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 3.0
• Labels: ignite-3
• Epic Link: New PageMemory Persistent Store
• Assignee: Philipp Shergalis
• Reporter: Kirill Tkalenko
• Created: 27/Jun/22 07:18
• Updated: 04/Sep/24 16:19

Description

When creating DelateFilePageStore at checkpoint, we sort the list of all dirty pages of the partition by pageIdx, write to disk the sorted list of pageIdx (for pageId -> pageIdx binary lookup), the contents of the dirty pages, and the current pageIdx of the page allocations.

I propose to optimize this a bit.
In DelateFilePageStore, store only changes in existing pages, and write all new pages immediately to FilePageStore, so we will reduce the work for the compacter (it will need to write less to the main partition file) and the sorted list of pageIdx will be smaller.

Since the allocation index becomes logical (which is stored in the FilePageStore) and depends on the first (newest) DelateFilePageStore, then if the checkpoint is not completed, we will not lose or break anything in the FilePageStore and on the new checkpoint we will write new pages on over of those that we write on the unfinished previous checkpoint.

Issue Links

• is a child of: IGNITE-17230 Support split-file page store