[IGNITE-20033] Implement local txnStateMap

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-beta2
• Labels: ignite-3 transaction3_recovery transactions
• Epic Link: Tx on unstable topology: commit partition recovery
• Assignee: Denis Chudov
• Reporter: Alexander Lapin
• Reviewer: Vladislav Pyatkov
• Created: 24/Jul/23 12:20
• Updated: 30/Aug/23 08:19
• Resolved: 30/Aug/23 08:19

Description

Motivation

For some purposes, in addition to persistent txnState in commit partition, it's required to have a txn meta on every transactional actor: txn coordinator, commit partition (both primary and all backups) and all enlisted partitions (both primary and backups). IEP-91 names such meta holder as txn state map and specifies following structure

txId -> (txState, txCoordAddr, commitTs)

Such map is originally designed to provide required information for writeIntentResolution: txState for local write intent resolution and txCoordAddr for coordinator-path one, but that's not all it's good for, so it definitely worth to implement it as soon as possible in order to unblock further activities.

-Why we have this ticket in "commit partition recovery" pack?

-Because in order to implement proper handling of a commit partition primary replica change (which is definitely a part of the "commit partition pack") it's required to:

1. Support local txnStateMap, in order to
2. Implement writeIntentResolution coordinator path, in order to
3. Calculate commitTimestamp on txn coordtinator istead of commit partition, where we do it now.

Definition of Done

Every transactional actor

• Txn Coordinator.
• CommitPartition, both primary and all backups.
• All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId -> (txState, txCoordAddr, commitTs)'.

 

Given map should be adjusted before any further transactional actions within node in a following way:

• Transaction coordinator.
  • On transaction start with state PENDING.
  • On transaction commit, right after commitTimestamp calucalttion with state FINISHING.
  • On txFinishReplicaResponse with either COMMITED or ABORTED.
• Commit partition.
  • On replica touch with state PENDING.
  • After succefull finishTxCommand application on majoirty with either COMMITED or ABORTED.
  • Seems that we don't need FINISHING state here, so let's skip it for now.
• Enlisted replica.
  • Primary
    • On replica touch with state PENDING.
    • On TxCleanupCommand COMMITED or ABORTED.
  • Backup
    • On touch replication PENDING.
    • On TxCleanupCommand COMMITED or ABORTED, same as for primary.

Implementation Notes

There are some open questions:

• Where to put this txnStateMap? TxManager?
• Properly handle same map multiple updates, based on the fact that Commit Partition is also an enlisted partition. To be honest I believe it's not that important. We may extent possible state change application rules to assume that null > PENDING, PENDING  > FINISHING, PENDING > COMMITED, PENDING > ABORTED, PENDING > PENDING, COMMITED > COMMITED and ABORTED > ABORTED are all possible.
• Eliminate excessive map updates in case of "one-phase commit". https://issues.apache.org/jira/browse/IGNITE-15927
• Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with txnStateMap.
• Definte "touch".
• It's reasonbale to use non-consistent node id as txCoordAddr because currently there's no sense in sending TxStateReq to the node that lost it's local txnStateMap because of node restart.

 

Issue Links

• blocks: IGNITE-20034 Implement writeIntentResolution coordinator path
• blocks: IGNITE-20041 Asynchronous writeIntent cleanup on write intent resolution
• duplicates: IGNITE-17638 Implement Txn state map to optimize write intent resolution
• links to: GitHub Pull Request #2483