[CASSANDRA-14156] [DTEST] [TRUNK] TestTopology.movement_test is flaky; fails assert "values not within 16.00% of the max: (851.41, 713.26)"

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Component/s: Test/dtest/python
• Labels: dtest
• Severity: Normal
• Assignee: Marcus Eriksson
• Reporter: Michael Kjellman

Authors

Marcus Eriksson

Reviewers

Jason Brown

• Created: 10/Jan/18 04:39
• Updated: 16/Apr/19 09:29
• Resolved: 27/Jul/18 07:44

Description

DTest* TestTopology.test_movement* is flaky. All of the testing so far (and thus all of the current known observed failures) have been when running against trunk. When the test fails, it always due to the assert_almost_equal assert.

AssertionError: values not within 16.00% of the max: (851.41, 713.26) ()

The following CircleCI runs are 2 examples with dtests runs that failed due to this test failing it's assert:
https://circleci.com/gh/mkjellman/cassandra/487
https://circleci.com/gh/mkjellman/cassandra/526

p.s. assert_almost_equal has a comment "@params error Optional margin of error. Default 0.16". I don't see any obvious notes for why the default is this magical 16% number. It looks like it was committed as part of a big bulk commit by Sean McCarthy (who I can't find on JIRA). If anyone has any history on the magic 16% allowed delta please share!