[IGNITE-18733] Mechanism to synchronize database schema changes between nodes

• Type: Epic
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0
• Component/s: persistence
• Labels: ignite-3
• Epic Name: Schema synchronization
• Ignite Flags: Docs Required, Release Notes Required
• Assignee: Unassigned
• Reporter: Sergey Chugunov
• Created: 07/Feb/23 08:12
• Updated: 04/Sep/24 16:19
• Resolved: 25/Dec/23 06:37

Description

Usability of Apache Ignite as a distributed database to a large extent depends on a solid support of dynamic schema changes.
DDL operations like adding of deleting columns of a table or (more importantly) adding or dropping indexes require proper syncronization between nodes and integration with a number of other components.

Regardless of implementation details synchronization mechanism:

• should allow (most) user operations and schema change operations to execute concurrently;
• should allow access to the data despite of the version of its schema;
• should shorten periods of time when certain types of operations are suspended or work ineffectively (e.g. because of rebuilding of an index).

Schema synchronization needs integration with the following components (the list may not be exhaustive):

• SQL engine - provide information about start and stop of operations, notify about [un]availability of certain indexes;
• TX engine - e.g. if a transaction is started concurrently with DDL operations;
• GC for MV storages - garbage collector needs to be aware of schema changes.

Issue Links

• fixes: IGNITE-20675 Thin 3.0: There is no guarantee to get the latest schema on schema change
• fixes: IGNITE-18536 Schema synchronization feature and Catalog feature integration
• is related to: IGNITE-20800 Mechanism to synchronize schema changes across a cluster: Part 2
• relates to: IGNITE-18449 Select from a table fails after the table is created