[CASSANDRA-10423] Paxos/LWT failures when moving node

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Duplicate
• Component/s: Feature/Lightweight Transactions
• Labels: LWT

Environment

Cassandra version: 2.0.14
Java-driver version: 2.0.11

• Severity: Normal
• Since Version: 2.0.14
• Assignee: Unassigned
• Reporter: Roger Schildmeijer
• Created: 30/Sep/15 15:18
• Updated: 25/Oct/19 13:11
• Resolved: 23/Dec/15 08:28

Description

While moving a node (nodetool move <newtoken>) we noticed that lwt started failing for some (~50%) requests. The java-driver (version 2.0.11) returned com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency SERIAL (7 replica were required but only 0 acknowledged the write). The cluster was not under heavy load.

I noticed that the failed lwt requests all took just above 1s. That information and the WriteTimeoutException could indicate that this happens:
https://github.com/apache/cassandra/blob/cassandra-2.0.14/src/java/org/apache/cassandra/service/StorageProxy.java#L268
I can't explain why though. Why would there be more cas contention just because a node is moving?

Issue Links

• duplicates: CASSANDRA-10887 Pending range calculator gives wrong pending ranges for moves
• relates to: CASSANDRA-10887 Pending range calculator gives wrong pending ranges for moves