[IGNITE-21084] Account for differences of logical part of now() on different nodes when waiting after a DDL

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0
• Labels: ignite-3
• Epic Link: Schema synchronization, Part 2
• Assignee: Roman Puchkovskiy
• Reporter: Roman Puchkovskiy
• Reviewer: Konstantin Orlov
• Created: 14/Dec/23 11:29
• Updated: 04/Sep/24 16:08
• Resolved: 15/Dec/23 08:51

Description

Client must see the results of a DDL they just executed, no matter through which node of the cluster a subsequent request is made. To maintain this, we must make sure that ActivationTimestamp(DDL) becomes non-future on all nodes of the cluster and only then can we complete the DDL future. As nodes' physical clocks might have some skew relatively to each other, we add MaxClockSkew to the timestamp till which we wait to compensate for this difference.

Analogously to HLC.now() being different because its physical part differs on different nodes, it might be different because logical parts are different on different nodes.

Let's assume we don't have any physical clock skew, and MaxClockSkew is 0. ActivationTimestamp(DDL) is (100, 5); (100 is physical part, 5 is logical part). We wait on node 1 (through which the DDL was executed) till its HLC.now() reaches (100, 5), then we complete the DDL future. The user goes to node 2 at which HLC.now() is (100, 2). The user executes a query at 'now', and the query sees the state before the DDL was executed, which breaks our requirement.

We can fix this by 'rounding' the timestamp-to-wait-for up in the following way:

1. If logical part is not 0, increment the physical part and set the logical part to 0
2. If the logical part is 0, leave the timestamp as is

As a result, for (100, 0) we will wait for (100, 0), and at node 1 HLC is at least (100, 0), so it cannot see the previous schema. For (100, 5) we would wait till (101, 0), which would also guarantee that a query executed after waiting sees the new schema version.

Issue Links

• is related to: IGNITE-21016 ItMixedQueriesTest.testIgniteSchemaAwaresAlterTableCommand is flaky
• relates to: IGNITE-21098 Increase additional wait after DDL by MaxClockSkew
• links to: GitHub Pull Request #2954