[CASSANDRA-16841] Unexpectedly ignored dtests

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.26, 3.11.12, 4.0.2, 4.1-alpha1, 4.1
• Component/s: Test/dtest/python
• Bug Category: Correctness
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: NA

Source Control Link

https://github.com/apache/cassandra-dtest/commit/6f3a4cb3c55323156c2b8686bea07a3362f3e861

Test and Documentation Plan

run tests

• Assignee: Ruslan Fomkin
• Reporter: Ruslan Fomkin

Authors

Ruslan Fomkin

Reviewers

Andres de la Peña, Ekaterina Dimitrova

• Created: 10/Aug/21 15:37
• Updated: 27/May/22 19:25
• Resolved: 14/Sep/21 11:21

Description

An issue, which I was hit:

When one class in a dtest file is marked as resource intensive, then all tests in all classes are treated as resource intensive. For example, repair_tests/repair_test.py contains three classes and the last class is marked as resource intensive:

@pytest.mark.resource_intensive
class TestRepairDataSystemTable(Tester):

So if I try to run an unmarked class: 

pytest --cassandra-dir=../cassandra repair_tests/repair_test.py::TestRepair --collect-only --skip-resource-intensive-tests

then all tests are ignored

collected 36 items / 36 deselected 

This is because a test is treated to be marked if any class in the same file has the mark. This bug was introduced in the fix of CASS-16399. Before only upgrade tests had such behaviour, i.e., if a class is marked as upgrade test, then all tests are upgrade test in the file.

 

This bug, for example, means that if the same file contains one class marked with vnodes and another class with no_vnodes, then no tests will be executed in the file.

I also noticed another issue that If a test run is executed with the argument -only-resource-intensive-tests and there is no sufficient resources for resource intensive tests, then no tests were executed. Thus it was necessary to provide -force-resource-intensive-tests in addition.

Suggestions for the solutions:

1. Require to mark each class and remove the special case of upgrade tests. This will simplify the implementation and might be more obvious for new comers.
2. Treat -only-resource-intensive-tests in the same way as -force-resource-intensive-tests, so it will be enough to just specify it even with no sufficient resources.

Update: comments were provided to keep only the first suggestion and do not implement the second suggestion. 

 

Issue Links

• links to: CircleCI
• links to: GH PR to cassandra-dtest trunk