Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-13583

test failure in rebuild_test.TestRebuild.disallow_rebuild_from_nonreplica_test

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: 4.0
    • Component/s: None

      Description

      example failure:

      http://cassci.datastax.com/job/trunk_novnode_dtest/524/testReport/rebuild_test/TestRebuild/disallow_rebuild_from_nonreplica_test

      Error Message
      
      ToolError not raised
      -------------------- >> begin captured logging << --------------------
      dtest: DEBUG: Python driver version in use: 3.10
      dtest: DEBUG: cluster ccm directory: /tmp/dtest-0tUjhX
      dtest: DEBUG: Done setting configuration options:
      {   'num_tokens': None,
          'phi_convict_threshold': 5,
          'range_request_timeout_in_ms': 10000,
          'read_request_timeout_in_ms': 10000,
          'request_timeout_in_ms': 10000,
          'truncate_request_timeout_in_ms': 10000,
          'write_request_timeout_in_ms': 10000}
      cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.3 DC1> discovered
      cassandra.cluster: INFO: New Cassandra host <Host: 127.0.0.2 DC1> discovered
      --------------------- >> end captured logging << ---------------------
      
      Stacktrace
      
        File "/usr/lib/python2.7/unittest/case.py", line 329, in run
          testMethod()
        File "/home/automaton/cassandra-dtest/tools/decorators.py", line 48, in wrappedtestrebuild
          f(obj)
        File "/home/automaton/cassandra-dtest/rebuild_test.py", line 357, in disallow_rebuild_from_nonreplica_test
          node1.nodetool('rebuild -ks ks1 -ts (%s,%s] -s %s' % (node3_token, node1_token, node3_address))
        File "/usr/lib/python2.7/unittest/case.py", line 116, in __exit__
          "{0} not raised".format(exc_name))
      
      1. node3.log
        24 kB
        Michael Hamm
      2. node3_gc.log
        20 kB
        Michael Hamm
      3. node3_debug.log
        115 kB
        Michael Hamm
      4. node2.log
        24 kB
        Michael Hamm
      5. node2_gc.log
        19 kB
        Michael Hamm
      6. node2_debug.log
        59 kB
        Michael Hamm
      7. node1.log
        26 kB
        Michael Hamm
      8. node1_gc.log
        21 kB
        Michael Hamm
      9. node1_debug.log
        97 kB
        Michael Hamm

        Activity

        Hide
        jkni Joel Knighton added a comment -

        This is consistently failing when run without vnodes after the commit of CASSANDRA-4650.

        Show
        jkni Joel Knighton added a comment - This is consistently failing when run without vnodes after the commit of CASSANDRA-4650 .
        Hide
        krummas Marcus Eriksson added a comment -

        https://github.com/krummas/cassandra/commits/marcuse/13583

        problem was that we ignored the sourceFound variable if localhost was in the map, but we need to still check that since localhost is filtered away when doing rebuild

        Show
        krummas Marcus Eriksson added a comment - https://github.com/krummas/cassandra/commits/marcuse/13583 problem was that we ignored the sourceFound variable if localhost was in the map, but we need to still check that since localhost is filtered away when doing rebuild
        Hide
        aweisberg Ariel Weisberg added a comment -

        So I don't quite have the context to understand this. For bootstrap we filter out localhost using an ISourceFilter for obvious reasons. But for rebuild we don't want to generate a source not found error if the localhost is the only source, but neither do we want to stream from it? I don't understand why we don't need a source to stream from for rebuild.

        Show
        aweisberg Ariel Weisberg added a comment - So I don't quite have the context to understand this. For bootstrap we filter out localhost using an ISourceFilter for obvious reasons. But for rebuild we don't want to generate a source not found error if the localhost is the only source, but neither do we want to stream from it? I don't understand why we don't need a source to stream from for rebuild.
        Hide
        aweisberg Ariel Weisberg added a comment -

        OK, I think I had it backwards. In the test case there is no source including localhost so we want the error and we previously didn't get it. But we also don't want to stream from localhost ever even if it counts as a source for some non-rebuild and non-bootstrap purpose.

        Do I have it right?

        Show
        aweisberg Ariel Weisberg added a comment - OK, I think I had it backwards. In the test case there is no source including localhost so we want the error and we previously didn't get it. But we also don't want to stream from localhost ever even if it counts as a source for some non-rebuild and non-bootstrap purpose. Do I have it right?
        Hide
        krummas Marcus Eriksson added a comment -
        Show
        krummas Marcus Eriksson added a comment - Do I have it right? yes running dtests here: https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/118/
        Hide
        aweisberg Ariel Weisberg added a comment -

        The dtest failures look unrelated so 1.

        Show
        aweisberg Ariel Weisberg added a comment - The dtest failures look unrelated so 1.
        Hide
        krummas Marcus Eriksson added a comment -

        committed, thanks

        Show
        krummas Marcus Eriksson added a comment - committed, thanks

          People

          • Assignee:
            krummas Marcus Eriksson
            Reporter:
            michael.hamm Michael Hamm
            Reviewer:
            Ariel Weisberg
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development