[CASSANDRA-16625] Add a CircleCI job to run some tests repeatedly

• Type: Task
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.25, 3.11.11, 4.0-rc2, 4.1-alpha1, 4.1
• Component/s: CI
• Change Category: Quality Assurance
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/a01cbfdce5e6b8959b792b3f693f87c2fc8f6b92

Test and Documentation Plan

The new CircleCI changes should be manually tested in the CircleCI UI.

• Assignee: Andres de la Peña
• Reporter: Andres de la Peña

Authors

Andres de la Peña

Reviewers

Berenguer Blasi, Ekaterina Dimitrova

• Created: 21/Apr/21 16:31
• Updated: 27/May/22 19:25
• Resolved: 12/May/21 11:24

Description

I think it could be useful to have an optional CircleCI job to run some specific tests n times. That way, tickets could attach CircleCI runs showing that the changes don't make a certain ticket flaky or, conversely, that they fix a flaky test. Doing this systematically should mitigate the risk of introducing new flaky tests, and I guess it would be more convenient and easy to share than running the tests locally or on a private CI system.

It would also be nice to have something similar in Jenkins, but I'm focusing this ticket on CircleCI because it's available also for non-committers, so assignees can run their tests before setting the tickets as ready for review.

Issue Links

• is a parent of: CASSANDRA-16654 Junit RepeatableRunner flaky tests helper
• relates to: CASSANDRA-16687 Support upgrade tests in CircleCI multiplexer
• links to: PR 3.0
• links to: PR 3.11
• links to: PR 4.0
• links to: PR trunk