[IGNITE-17615] Close local cursors on primary replica expiration

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0
• Labels: ignite-3
• Epic Link: Txn cursors vacuum
• Assignee: Vladislav Pyatkov
• Reporter: Sergey Uttsel
• Reviewer: Alexander Lapin
• Created: 02/Sep/22 07:40
• Updated: 04/Sep/24 16:19
• Resolved: 15/Jan/24 14:34

Description

Motivation

According to our tx protocol, it’s impossible to commit a transaction if any of the enlisted primary replicas have expired. It also means that there’s no sense in preserving tx related volatile state such as locks and cursors. Pay attention, that it’s still useful to preserve txnState in the txnStateLocalMap because it will ease write intent resolution procedure. Locks release on primary replica expiration was already implemented, so this ticket is only about closing cursors on primary expiration.

Definition of Done

• On primary replica expiration all RW-scoped cursors are closed.

Implementation Notes

1.In `org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#onPrimaryExpired` we release all tx locks

futs.add(allOf(txFuts).whenComplete((unused, throwable) -> releaseTxLocks(txId)));

Seems reasonable to reuse same event to close the cursors. Worth mentioning that given action should be asynchronous. I believe that we may do the cursors close in partition striped pool. See StripedThreadPoolExecutor for more details. Another option here is to introduce special dedicated cleanup thread and use it instead. That will be a part of TX Resourse Cleanup design.

2. So, that was about when to close, let’s clarify what to close. Seems that it’s trivial. We have `org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#cursors` right in partition replica listeners. We even have corresponding helper method `org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#closeAllTransactionCursors`

 

All in all, seems that it's required to substitute

 

futs.add(allOf(txFuts).whenComplete((unused, throwable) -> releaseTxLocks(txId)));

with

                    futs.add(allOf(txFuts).whenComplete((unused, throwable) -> {
                        releaseTxLocks(txId);
                        try {
                            closeAllTransactionCursors(txId);
                        } catch (Exception e) {
                            LOG.warn("Unable to close cursor on primary replica expiration", e);
                        }
                    }));

Tests are trickey though.

 

Issue Links

• is related to: IGNITE-20055 Durable txCleanupReplicaRequest send from the commit partition
• is related to: IGNITE-20700 Implement durable transaction coordinator finish
• links to: GitHub Pull Request #3001