[IGNITE-20637] Implement recovery of distributed index building

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0
• Labels: ignite-3
• Epic Link: Dynamic indexes creation
• Assignee: Kirill Tkalenko
• Reporter: Kirill Tkalenko
• Reviewer: Roman Puchkovskiy
• Created: 12/Oct/23 15:37
• Updated: 04/Sep/24 16:10
• Resolved: 31/Oct/23 16:25

Description

We need to take care of recovery for the distributed index construction mechanism:

• For indexes that are not available for read-write:

1. Make sure that the corresponding keys are present in the metastore: startBuildIndex.<indexId> and partitionBuildIndex.<indexId>.<partitionId_0>...partitionBuildIndex.<indexId>.<partitionId_N>.
   Notest:
   This is easy to implement, since there is an auxiliary key startBuildIndex.<indexId>, which, in its absence, will allow you to accurately understand that there are no keys for this index in metasore.
2. For the build distributed index of a specific partition, delete key partitionBuildIndex.<indexId>.<partitionId> in the metastore.
   Notest:
   This point is probably the most difficult and requires thought before implementation, since it will most likely require raising a replication group and applying a replication log.
3. If there are no keys partitionBuildIndex.<indexId>.<partitionId_0>...partitionBuildIndex.<indexId>.<partitionId_N> in the metastore, execute org.apache.ignite.internal.catalog.commands.MakeIndexAvailableCommand.

• For indexes that are available for read-write:

1. Delete key startBuildIndex.<indexId> in the metastore if it remains.

Issue Links

• is blocked by: IGNITE-19276 Implement a mechanism to build indices distributively
• links to: GitHub Pull Request #2756