[CASSANDRA-16693] Flaky TestPaxos.test_replica_availability

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0-rc2, 4.0, 4.1-alpha1, 4.1
• Component/s: Test/dtest/python
• Bug Category: Correctness
• Severity: Normal
• Complexity: Normal
• Discovered By: Unit Test
• Platform: All
• Since Version: 4.0

Source Control Link

https://github.com/apache/cassandra-dtest/commit/c67b12d14d75a70336b7b7fef8d4f81d80e32124

Test and Documentation Plan

See PR

• Assignee: Berenguer Blasi
• Reporter: Berenguer Blasi

Authors

Berenguer Blasi

Reviewers

Ekaterina Dimitrova

• Created: 25/May/21 06:43
• Updated: 27/May/22 19:25
• Resolved: 26/May/21 05:35

Description

Flaky TestPaxos.test_replica_availability

Error Message

cassandra.Unavailable: Error from server: code=1000 [Unavailable exception] message="Cannot achieve consistency level SERIAL" info={'consistency': 'SERIAL', 'required_replicas': 2, 'alive_replicas': 1}

Stacktrace

self = <paxos_test.TestPaxos object at 0x7ff6f83299d0>

    def test_replica_availability(self):
        """
            @jira_ticket CASSANDRA-8640
    
            Regression test for a bug (CASSANDRA-8640) that required all nodes to
            be available in order to run LWT queries, even if the query could
            complete correctly with quorum nodes available.
            """
        session = self.prepare(nodes=3, rf=3)
        session.execute("CREATE TABLE test (k int PRIMARY KEY, v int)")
        session.execute("INSERT INTO test (k, v) VALUES (0, 0) IF NOT EXISTS")
    
        self.cluster.nodelist()[2].stop()
        session.execute("INSERT INTO test (k, v) VALUES (1, 1) IF NOT EXISTS")
    
        self.cluster.nodelist()[1].stop()
        assert_unavailable(session.execute, "INSERT INTO test (k, v) VALUES (2, 2) IF NOT EXISTS")
    
        self.cluster.nodelist()[1].start()
>       session.execute("INSERT INTO test (k, v) VALUES (3, 3) IF NOT EXISTS")

paxos_test.py:58: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
../venv/src/cassandra-driver/cassandra/cluster.py:2618: in execute
    return self.execute_async(query, parameters, trace, custom_payload, timeout, execution_profile, paging_state, host, execute_as).result()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <ResponseFuture: query='<SimpleStatement query="INSERT INTO test (k, v) VALUES (3, 3) IF NOT EXISTS", consistency=Not ...el SERIAL" info={'consistency': 'SERIAL', 'required_replicas': 2, 'alive_replicas': 1} coordinator_host=127.0.0.1:9042>

    def result(self):
        """
            Return the final result or raise an Exception if errors were
            encountered.  If the final result or error has not been set
            yet, this method will block until it is set, or the timeout
            set for the request expires.
    
            Timeout is specified in the Session request execution functions.
            If the timeout is exceeded, an :exc:`cassandra.OperationTimedOut` will be raised.
            This is a client-side timeout. For more information
            about server-side coordinator timeouts, see :class:`.policies.RetryPolicy`.
    
            Example usage::
    
                >>> future = session.execute_async("SELECT * FROM mycf")
                >>> # do other stuff...
    
                >>> try:
                ...     rows = future.result()
                ...     for row in rows:
                ...         ... # process results
                ... except Exception:
                ...     log.exception("Operation failed:")
    
            """
        self._event.wait()
        if self._final_result is not _NOT_SET:
            return ResultSet(self, self._final_result)
        else:
>           raise self._final_exception
E           cassandra.Unavailable: Error from server: code=1000 [Unavailable exception] message="Cannot achieve consistency level SERIAL" info={'consistency': 'SERIAL', 'required_replicas': 2, 'alive_replicas': 1}

../venv/src/cassandra-driver/cassandra/cluster.py:4894: Unavailable

Issue Links

• links to: CASSANDRA-16693 PR