[IGNITE-20108] Use tableEnlistTs as baseTs when validating schema

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 3.0
• Labels: iep-110 ignite-3
• Epic Link: Schema synchronization, Part 3
• Assignee: Unassigned
• Reporter: Roman Puchkovskiy
• Created: 01/Aug/23 07:30
• Updated: 04/Sep/24 16:10

Description

This design is problematic

This might lead to the following:

1. A transaction is started
2. It touches table A
3. Column c1 is added to tables A and B
4. The transactin touches table B
5. Now, it sees old version of A without A.c1, but new version of B with B.c1, which seems to be inconsistent

Probably this has to be redesigned (or at least examined carefully). There is an alternative in IGNITE-21003.

The design

https://cwiki.apache.org/confluence/display/IGNITE/IEP-110%3A+Schema+synchronization%3A+basic+schema+changes#IEP110:Schemasynchronization:basicschemachanges-Basicrequirements requires an ALTER TABLE occurring after a transaction had started but before the tx touched the table for the first time to be visible to the tx and not cause its abortion.

To achieve this, we should take baseTs=tableEnlistTs(tx, table).

Both tx coordinator and primary need to track those tableEnlistTs. On the coordinator and the primary these values will most likely be different (because they are taken from different clocks: the coordinator clock and the primary clock), but this is acceptable.

Coordinator

The coordinator already tracks some information about a transaction (namely, its enlisted partitions); tableEnlistTs should be added to this structure. When doing the client schema version check (IGNITE-20106), the coordinator should check whether tableEnlistTs already exists for the table in question, and, if not, generate it (as HLC.now()) and remember it; then use it to obtain the server-side schema version and use the version to validate the client's request.

Primary

The primary should track tableEnlistSchemaVersion too. It should make sure that the coordinator's tableEnlistSchemaVersion (that can be determined by the schema version of the tuples that the client sends) matches the primary's tableEnlistSchemaVersion (that is determined by the operationTs (on the primary clock) of the first operation touching the table in the transaction.

On recovery, the primary should recover tableEnlistSchemaVersions; it can do so by scanning all writes made by a transaction and looking into Schema Version mentioned in the corresponding tuples.

Issue Links

• depends upon: IGNITE-20107 Make table created after tx started visible to the tx
• is related to: IGNITE-21003 Switch to the 'always use current schema' approach