Description
While handling SupplyMessage, node handles each supplied data entry separately, which causes a WAL record for each entry to be written. It significantly limits preloading speed.
We can improve rebalancing speed and reduce pressure on disk by disabling WAL until all data is loaded. The disadvantage of this approach is that data might get corrupted if node crashes - but node that crashed during preloading has to clear all it's data anyway. However, it is important to distinguish situations when new node joined cluster or added to baseline topology (and doesn't hold any data) and when additional partitions got assigned to node after baseline topology changed (in this case node has to keep all data in consistent state).
Attachments
Issue Links
- is related to
-
IGNITE-7934 Do not checkpoint pages from MOVING partitions
-
- Open
-
- links to
