[IGNITE-18694] Recovery for DistributionZoneRebalanceEngine#metaStorageManager on DistributionZoneManager#start()

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: dzm-reviewed ignite-3 tech-debt
• Assignee: Unassigned
• Reporter: Sergey Uttsel
• Created: 01/Feb/23 18:01
• Updated: 17/Oct/23 09:46

Description

Motivation

DistributionZoneRebalanceEngine#dataNodesListener processes events with zones' data nodes updates and invokes RebalanceUtil#updatePendingAssignmentsKeys with new data nodes value. updatePendingAssignmentsKeys does async metaStorageMgr#invoke. It's possible that dataNodesListener processed data nodes event then the nodes crashed without updating assignments in metastorage.

Implementation Notes

To fix it we can redo all logic from `createDistributionZonesDataNodesListener()`. On DistributionZoneManager#start we need to read from `vault` data nodes for all zones and invoke `updatePendingAssignmentsKeys` for all tables with `metaStorageManager.appliedRevision()`. If the last event with data nodes has not updated pending assignments then assignments will be updated. If the last event with data nodes have updated pending assignments then an update assignments will be invoked another time but it will not update assignments, because there is a check for case when new assignments equals to old one.

Definition of Done

Created a recovery for this case.

Issue Links

• depends upon: IGNITE-19967 Design restart of Distribution Zone Manager in accordance with the new local recovery process
• is a child of: IGNITE-17924 Core distributions zones functionality.