[IGNITE-17267] Preparing a checkpoint for a DeltaFilePageStore

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-beta1
• Labels: ignite-3
• Epic Link: New PageMemory Persistent Store
• Assignee: Kirill Tkalenko
• Reporter: Kirill Tkalenko
• Reviewer: Roman Puchkovskiy
• Created: 29/Jun/22 09:38
• Updated: 13/Oct/22 12:01
• Resolved: 07/Jul/22 08:13

Description

To implement the DeltaFilePageStore, we need to prepare a checkpoint for their creation.

What should be done:

1. Sort dirty pages (FullPageId) by groupId, partitionId, pageIdx;
2. In several threads, write dirty pages in batches for each partition and know the number of these pages.

What can we look at:

• org.apache.ignite.internal.pagememory.persistence.checkpoint.CheckpointWorkflow#beginCheckpoint
• org.apache.ignite.internal.pagememory.persistence.checkpoint.CheckpointWorkflow#splitAndSortCheckpointPagesIfNeeded
• org.apache.ignite.internal.pagememory.persistence.checkpoint.Checkpointer#writePages

Some implementation notes:

• We can replace CheckpointWorkflow#parallelSortThreshold with a constant at 40k, according to the paper.

Issue Links

• is related to: IGNITE-17230 Support split-file page store
• links to: GitHub Pull Request #907
• links to: Threshold for parallel sort