[CASSANDRA-18947] Test failure: dtest-novnode.disk_balance_test.TestDiskBalance.test_disk_balance_stress

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.0-rc1, 5.0
• Component/s: Test/dtest/python
• Bug Category: Correctness - Test Failure
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: 4.0

Source Control Link

https://github.com/apache/cassandra-dtest/commit/365085bbd76ee717e265598fd83c6f4c39e1f1e6

Test and Documentation Plan

Pending approach approval

• Assignee: Berenguer Blasi
• Reporter: Ekaterina Dimitrova

Authors

Berenguer Blasi

Reviewers

Ekaterina Dimitrova, Michael Semb Wever

• Created: 20/Oct/23 17:51
• Updated: 18/Dec/23 08:58
• Resolved: 01/Dec/23 06:14

Description

Seen here:

https://ci-cassandra.apache.org/job/Cassandra-5.0/72/testReport/dtest-novnode.disk_balance_test/TestDiskBalance/test_disk_balance_stress/

 

Error Message
AssertionError: values not within 10.00% of the max: (2534183, 2762123, 2423706) (node1)

Stacktrace
self = <disk_balance_test.TestDiskBalance object at 0x7f8fae9b7590> def test_disk_balance_stress(self): cluster = self.cluster if self.dtest_config.use_vnodes: cluster.set_configuration_options(values={'num_tokens': 256}) cluster.populate(4).start() node1 = cluster.nodes['node1'] node1.stress(['write', 'n=50k', 'no-warmup', '-rate', 'threads=100', '-schema', 'replication(factor=3)', 'compaction(strategy=SizeTieredCompactionStrategy,enabled=false)']) cluster.flush() # make sure the data directories are balanced: for node in cluster.nodelist(): > self.assert_balanced(node) disk_balance_test.py:48: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ disk_balance_test.py:186: in assert_balanced assert_almost_equal(*new_sums, error=0.1, error_message=node.name) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ args = (2534183, 2762123, 2423706) kwargs = {'error': 0.1, 'error_message': 'node1'}, error = 0.1, vmax = 2762123 vmin = 2423706, error_message = 'node1' def assert_almost_equal(*args, **kwargs): """ Assert variable number of arguments all fall within a margin of error. @params *args variable number of numerical arguments to check @params error Optional margin of error. Default 0.16 @params error_message Optional error message to print. Default '' Examples: assert_almost_equal(sizes[2], init_size) assert_almost_equal(ttl_session1, ttl_session2[0][0], error=0.005) """ error = kwargs['error'] if 'error' in kwargs else 0.16 vmax = max(args) vmin = min(args) error_message = '' if 'error_message' not in kwargs else kwargs['error_message'] assert vmin > vmax * (1.0 - error) or vmin == vmax, \ > "values not within {:.2f}% of the max: {} ({})".format(error * 100, args, error_message) E AssertionError: values not within 10.00% of the max: (2534183, 2762123, 2423706) (node1) tools/assertions.py:206: AssertionError

 

Issue Links

• relates to: CASSANDRA-16196 Fix flaky test test_disk_balance_after_boundary_change_lcs - disk_balance_test.TestDiskBalance
• links to: CASSANDRA-18947-5.0 PR