[SUREFIRE-1903] Tests count results are collated incorrectly when using rerun feature

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-M5
• Component/s: Maven Surefire Plugin, Maven Surefire Report Plugin
• Assignee: Unassigned
• Reporter: Michal Toth
• Created: 26/Mar/21 13:23
• Updated: 26/Mar/21 13:23

Description

There is a discrepancy in test output results when we execute some of our tests and some of them are rerun.
For example: We executed 1 test class, containing 7 tests and here are results:

mvn -f /home/cloud-user/jenkins/workspace/strimzi/strimzi-pull-request-job/systemtest/pom.xml -P all verify .... -Dfailsafe.rerunFailingTestsCount=2 
...
[ERROR] Tests run: 7, Failures: 0, Errors: 2, Skipped: 1, Time elapsed: 1,062.97 s <<< FAILURE! - in io.strimzi.systemtest.log.LogSettingST
...
[ERROR] Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 708.103 s <<< FAILURE! 
...
Overall result
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 434.379 s - in io.strimzi.systemtest.log.LogSettingST

I also found a similar discussion PR at https://github.com/apache/maven-surefire/pull/313

Thank you