[CASSANDRA-12904] dtest failure in repair_tests.repair_test.TestRepair.simple_sequential_repair_test

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Auto Closed
• Component/s: Test/dtest/python
• Labels: dtest test-failure
• Severity: Normal
• Assignee: Yuki Morishita
• Reporter: Michael Shuler

Authors

Yuki Morishita

• Created: 11/Nov/16 17:38
• Updated: 13/Sep/21 17:12
• Resolved: 13/Sep/21 17:12

Description

example failure:

http://cassci.datastax.com/job/cassandra-2.2_dtest/710/testReport/repair_tests.repair_test/TestRepair/simple_sequential_repair_test/

Error Message

2001 != 2000
-------------------- >> begin captured logging << --------------------
dtest: DEBUG: cluster ccm directory: /tmp/dtest-gzVU4X
dtest: DEBUG: Done setting configuration options:
{   'initial_token': None,
    'num_tokens': '32',
    'phi_convict_threshold': 5,
    'range_request_timeout_in_ms': 10000,
    'read_request_timeout_in_ms': 10000,
    'request_timeout_in_ms': 10000,
    'truncate_request_timeout_in_ms': 10000,
    'write_request_timeout_in_ms': 10000}
dtest: DEBUG: Starting cluster..
cassandra.policies: INFO: Using datacenter 'datacenter1' for DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify a local_dc to the constructor, or limit contact points to local cluster nodes
cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.3 datacenter1> discovered
cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered
dtest: DEBUG: Inserting data...
cassandra.cluster: WARNING: Host 127.0.0.3 has been marked down
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 2.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 4.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused
cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 8.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused
cassandra.pool: INFO: Successful reconnection to 127.0.0.3, marking node up if it isn't already
cassandra.cluster: INFO: Host 127.0.0.3 may be up; will prepare queries and open connection pool
cassandra.cluster: INFO: Connection pools established for node 127.0.0.3
dtest: DEBUG: Checking data on node3...
cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.1 datacenter1> discovered
cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered
--------------------- >> end captured logging << ---------------------
Stacktrace

  File "/usr/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 291, in simple_sequential_repair_test
    self._simple_repair(sequential=True)
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 386, in _simple_repair
    self._repair_and_verify(sequential)
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 118, in _repair_and_verify
    self.check_rows_on_node(node3, 2000, missings=[1000])
  File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 71, in check_rows_on_node
    self.assertEqual(len(result), rows)
  File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
    raise self.failureException(msg)
"2001 != 2000\n-------------------- >> begin captured logging << --------------------\ndtest: DEBUG: cluster ccm directory: /tmp/dtest-gzVU4X\ndtest: DEBUG: Done setting configuration options:\n{   'initial_token': None,\n    'num_tokens': '32',\n    'phi_convict_threshold': 5,\n    'range_request_timeout_in_ms': 10000,\n    'read_request_timeout_in_ms': 10000,\n    'request_timeout_in_ms': 10000,\n    'truncate_request_timeout_in_ms': 10000,\n    'write_request_timeout_in_ms': 10000}\ndtest: DEBUG: Starting cluster..\ncassandra.policies: INFO: Using datacenter 'datacenter1' for DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify a local_dc to the constructor, or limit contact points to local cluster nodes\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.3 datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered\ndtest: DEBUG: Inserting data...\ncassandra.cluster: WARNING: Host 127.0.0.3 has been marked down\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 2.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 4.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 8.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused\ncassandra.pool: INFO: Successful reconnection to 127.0.0.3, marking node up if it isn't already\ncassandra.cluster: INFO: Host 127.0.0.3 may be up; will prepare queries and open connection pool\ncassandra.cluster: INFO: Connection pools established for node 127.0.0.3\ndtest: DEBUG: Checking data on node3...\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.1 datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered\n--------------------- >> end captured logging << ---------------------"