Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-10423

Paxos/LWT failures when moving node

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Duplicate
    • Fix Version/s: None
    • Labels:
    • Environment:

      Cassandra version: 2.0.14
      Java-driver version: 2.0.11

    • Severity:
      Normal
    • Since Version:

      Description

      While moving a node (nodetool move <newtoken>) we noticed that lwt started failing for some (~50%) requests. The java-driver (version 2.0.11) returned com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency SERIAL (7 replica were required but only 0 acknowledged the write). The cluster was not under heavy load.

      I noticed that the failed lwt requests all took just above 1s. That information and the WriteTimeoutException could indicate that this happens:
      https://github.com/apache/cassandra/blob/cassandra-2.0.14/src/java/org/apache/cassandra/service/StorageProxy.java#L268
      I can't explain why though. Why would there be more cas contention just because a node is moving?

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rschildmeijer Roger Schildmeijer
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: