[IGNITE-21284] Internal API for manual raft group configuration update

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0
• Labels: ignite-3
• Epic Link: Ignite 3 Disaster Recovery
• Assignee: Ivan Bessonov
• Reporter: Ivan Bessonov
• Reviewer: Kirill Tkalenko
• Created: 17/Jan/24 12:19
• Updated: 04/Sep/24 16:07
• Resolved: 05/Mar/24 09:16

Description

We need an API (with implementation) that's analogous to "reset-lost-partitions", but with the ability to reuse living minority of nodes.

This API should gather the states of partitions, identify healthy peers, and use them as a new raft group configuration (through the update of assignments).

We have to make sure that node with latest log index will become a leader, so we will have to propagate desired minimum for log index in assignments and use it during the voting.

What's implemented

"resetPartitions" operation in distributed zone manager. It identifies partitions where only a minority of nodes is online (thus they won't be able to execute "changePeersAsync"), and writes a "forced pending assignments" for them.

Forced assignment excludes stable nodes, that are not present in pending assignment, from a new raft group configuration. It also performs a "resetPeers" operation on alive nodes from the stable assignment.

Complete loss of all nodes from stable assignments is not yet implemented, at least one node is required to be elected as a leader.

Issue Links

• blocks: IGNITE-21295 REST API for manual raft group configuration update
• links to: GitHub Pull Request #3261