Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-12904

dtest failure in repair_tests.repair_test.TestRepair.simple_sequential_repair_test

    Details

    • Type: Bug
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Fix Version/s: None
    • Component/s: Test/dtest
    • Severity:
      Normal

      Description

      example failure:

      http://cassci.datastax.com/job/cassandra-2.2_dtest/710/testReport/repair_tests.repair_test/TestRepair/simple_sequential_repair_test/

      Error Message
      
      2001 != 2000
      -------------------- >> begin captured logging << --------------------
      dtest: DEBUG: cluster ccm directory: /tmp/dtest-gzVU4X
      dtest: DEBUG: Done setting configuration options:
      {   'initial_token': None,
          'num_tokens': '32',
          'phi_convict_threshold': 5,
          'range_request_timeout_in_ms': 10000,
          'read_request_timeout_in_ms': 10000,
          'request_timeout_in_ms': 10000,
          'truncate_request_timeout_in_ms': 10000,
          'write_request_timeout_in_ms': 10000}
      dtest: DEBUG: Starting cluster..
      cassandra.policies: INFO: Using datacenter 'datacenter1' for DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify a local_dc to the constructor, or limit contact points to local cluster nodes
      cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.3 datacenter1> discovered
      cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered
      dtest: DEBUG: Inserting data...
      cassandra.cluster: WARNING: Host 127.0.0.3 has been marked down
      cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 2.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused
      cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 4.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused
      cassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 8.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused
      cassandra.pool: INFO: Successful reconnection to 127.0.0.3, marking node up if it isn't already
      cassandra.cluster: INFO: Host 127.0.0.3 may be up; will prepare queries and open connection pool
      cassandra.cluster: INFO: Connection pools established for node 127.0.0.3
      dtest: DEBUG: Checking data on node3...
      cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.1 datacenter1> discovered
      cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered
      --------------------- >> end captured logging << ---------------------
      Stacktrace
      
        File "/usr/lib/python2.7/unittest/case.py", line 329, in run
          testMethod()
        File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 291, in simple_sequential_repair_test
          self._simple_repair(sequential=True)
        File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 386, in _simple_repair
          self._repair_and_verify(sequential)
        File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 118, in _repair_and_verify
          self.check_rows_on_node(node3, 2000, missings=[1000])
        File "/home/automaton/cassandra-dtest/repair_tests/repair_test.py", line 71, in check_rows_on_node
          self.assertEqual(len(result), rows)
        File "/usr/lib/python2.7/unittest/case.py", line 513, in assertEqual
          assertion_func(first, second, msg=msg)
        File "/usr/lib/python2.7/unittest/case.py", line 506, in _baseAssertEqual
          raise self.failureException(msg)
      "2001 != 2000\n-------------------- >> begin captured logging << --------------------\ndtest: DEBUG: cluster ccm directory: /tmp/dtest-gzVU4X\ndtest: DEBUG: Done setting configuration options:\n{   'initial_token': None,\n    'num_tokens': '32',\n    'phi_convict_threshold': 5,\n    'range_request_timeout_in_ms': 10000,\n    'read_request_timeout_in_ms': 10000,\n    'request_timeout_in_ms': 10000,\n    'truncate_request_timeout_in_ms': 10000,\n    'write_request_timeout_in_ms': 10000}\ndtest: DEBUG: Starting cluster..\ncassandra.policies: INFO: Using datacenter 'datacenter1' for DCAwareRoundRobinPolicy (via host '127.0.0.1'); if incorrect, please specify a local_dc to the constructor, or limit contact points to local cluster nodes\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.3 datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered\ndtest: DEBUG: Inserting data...\ncassandra.cluster: WARNING: Host 127.0.0.3 has been marked down\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 2.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 4.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused\ncassandra.pool: WARNING: Error attempting to reconnect to 127.0.0.3, scheduling retry in 8.0 seconds: [Errno 111] Tried connecting to [('127.0.0.3', 9042)]. Last error: Connection refused\ncassandra.pool: INFO: Successful reconnection to 127.0.0.3, marking node up if it isn't already\ncassandra.cluster: INFO: Host 127.0.0.3 may be up; will prepare queries and open connection pool\ncassandra.cluster: INFO: Connection pools established for node 127.0.0.3\ndtest: DEBUG: Checking data on node3...\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.1 datacenter1> discovered\ncassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 datacenter1> discovered\n--------------------- >> end captured logging << ---------------------"
      

        Attachments

        1. node3.log
          82 kB
          Michael Shuler
        2. node3_gc.log
          31 kB
          Michael Shuler
        3. node3_debug.log
          303 kB
          Michael Shuler
        4. node2.log
          62 kB
          Michael Shuler
        5. node2_gc.log
          54 kB
          Michael Shuler
        6. node2_debug.log
          190 kB
          Michael Shuler
        7. node1.log
          61 kB
          Michael Shuler
        8. node1_gc.log
          44 kB
          Michael Shuler
        9. node1_debug.log
          175 kB
          Michael Shuler

          Activity

            People

            • Assignee:
              yukim Yuki Morishita
              Reporter:
              mshuler Michael Shuler
              Authors:
              Yuki Morishita
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: