[CASSANDRA-18499] Test Failure: upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.0-alpha1, 5.0
• Component/s: Test/dtest/python
• Bug Category: Correctness - Test Failure
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: 5.0

Source Control Link

https://github.com/apache/cassandra-dtest/commit/1da18efb2ffd3f9efc3b8b178b2a8d38a6831056 https://github.com/apache/cassandra/commit/6643ea7551d904383af6d036e18b8a66af0ceba2

Test and Documentation Plan

ci

• Assignee: Michael Semb Wever
• Reporter: Andres de la Peña

Authors

Michael Semb Wever

Reviewers

Brandon Williams, Jacek Lewandowski

• Created: 04/May/23 15:13
• Updated: 12/Sep/23 13:02
• Resolved: 20/Jul/23 16:44

Description

The upgrade Python dtest upgrade_through_versions_test.py::TestUpgrade::test_rolling_upgrade_with_internode_ssl seems to fail on CircleCI at least for trunk:

• https://app.circleci.com/pipelines/github/adelapena/cassandra/2882/workflows/b9abc2b2-2e79-47b2-b7ce-c549e75293bc/jobs/42698/tests

self = <upgrade_tests.upgrade_through_versions_test.TestProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD object at 0x7f89fc0f17f0>

    @pytest.mark.timeout(3000)
    def test_rolling_upgrade_with_internode_ssl(self):
        """
        Rolling upgrade test using internode ssl.
        """
>       self.upgrade_scenario(rolling=True, internode_ssl=True)

upgrade_tests/upgrade_through_versions_test.py:371: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <upgrade_tests.upgrade_through_versions_test.TestProtoV3Upgrade_AllVersions_EndsAt_Trunk_HEAD object at 0x7f89fc0f17f0>
populate = True, create_schema = True, rolling = True, after_upgrade_call = ()
internode_ssl = True

    def upgrade_scenario(self, populate=True, create_schema=True, rolling=False, after_upgrade_call=(), internode_ssl=False):
        # Record the rows we write as we go:
        if populate:
            self.prepare()
        self.row_values = set()
        cluster = self.cluster
        if cluster.version() >= '3.0':
            cluster.set_configuration_options({'enable_user_defined_functions': 'true',
                                               'enable_scripted_user_defined_functions': 'true'})
        elif cluster.version() >= '2.2':
            cluster.set_configuration_options({'enable_user_defined_functions': 'true'})
    
        if internode_ssl:
            logger.debug("***using internode ssl***")
            generate_ssl_stores(self.fixture_dtest_setup.test_path)
            self.cluster.enable_internode_ssl(self.fixture_dtest_setup.test_path)
    
        if populate:
            # Start with 3 node cluster
            logger.debug('Creating cluster (%s)' % self.test_version_metas[0].version)
            cluster.populate(3)
            [node.start(use_jna=True, wait_for_binary_proto=True) for node in cluster.nodelist()]
        else:
            logger.debug("Skipping cluster creation (should already be built)")
    
        # add nodes to self for convenience
        for i, node in enumerate(cluster.nodelist(), 1):
            node_name = 'node' + str(i)
            setattr(self, node_name, node)
    
        if create_schema:
            if rolling:
                self._create_schema_for_rolling()
            else:
                self._create_schema()
        else:
            logger.debug("Skipping schema creation (should already be built)")
    
        self._log_current_ver(self.test_version_metas[0])
    
        if rolling:
            # start up processes to write and verify data
            write_proc, verify_proc, verification_queue = self._start_continuous_write_and_verify(wait_for_rowcount=5000)
    
            # upgrade through versions
            for version_meta in self.test_version_metas[1:]:
                if version_meta.family > '3.11' and internode_ssl:
                    seeds =[]
                    for seed in cluster.seeds:
>                       seeds.append(seed.ip_addr + ':7001')
E                       AttributeError: 'str' object has no attribute 'ip_addr'

upgrade_tests/upgrade_through_versions_test.py:422: AttributeError

I haven't seen this failure on Jenkins yet.

Issue Links

• Discovered while testing: CASSANDRA-8720 Provide tools for finding wide row/partition keys
• is related to: CASSANDRA-18567 Add jobs for resource-intensive Python upgrade tests
• relates to: CASSANDRA-13107 Remove -XX:ThreadPriorityPolicy=42 jvm flag
• relates to: CASSANDRA-17869 Add JDK17 option to cassandra-builds (build-scripts and jenkins dsl) and on jenkins agents