[CASSANDRA-15745] Conflicting LWT transactions may be committed during topology change

• Type: Bug
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Component/s: Feature/Lightweight Transactions
• Bug Category: Correctness - Unrecoverable Corruption / Loss
• Severity: Normal
• Complexity: Challenging
• Discovered By: User Report
• Platform: All
• Assignee: Unassigned
• Reporter: Konstantin
• Created: 21/Apr/20 13:30
• Updated: 14/May/20 11:55

Description

Let's consider a cluster which consists of replicas A, B and C.
We're adding replica D which replaces A.

A scenario is possible when two conflicting transactions, CAS1 and CAS2, may be committed during replace:

CAS2 ballot > CAS1 ballot
CAS2 and CAS1 conflict on LWT condition, yet both of them may be committed in case of the following sequence of events:

Topology change starts, advertises on C
CAS1 starts on node C, uses

{A, B, C, D}

CAS2 starts on node A, still uses

{A, B, C}

Topology change is advertised on A
CAS1 prepares on

{B, C, D}

CAS2 prepares and accepts on

{A, B}

, commits on A
CAS1 accepts on D, then stops
Streaming starts, topology change finishes, A is removed
CAS3 prepares using C and D. It sees the accept of CAS1 and replays it
Both CAS1 and CAS2 are committed.