[SUREFIRE-2065] Test Reports Inconsistencies with Parameterized and junit4

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-M5, 3.0.0-M6, 3.0.0-M7
• Fix Version/s: 3.0.0-M9
• Component/s: Junit 4.x support
• Assignee: Olivier Lamy
• Reporter: Christian Marquez Grabia
• Created: 08/Apr/22 17:52
• Updated: 16/Feb/23 18:52
• Resolved: 06/Feb/23 23:32

Description

Running parameterized tests with the junit-platform provider causes tests to be counted as single test (unlike using junit4 provider). This makes the tests to be marked as 'Flaky' and marks the build as successful if the retry failed count is greater than zero. Reports also show a single test run even if there were multiple tests with different parameters.

For a sample project with the problem see surefire-flaky-issue 

This is also a possibly related to issue SUREFIRE-2010 and may help in the resolution of it.

Issue Links

• is duplicated by: SUREFIRE-2087 rerunFailingTestsCount incorrectly marks failed parametrized test as pass
• links to: GitHub Pull Request #516
• links to: GitHub Pull Request #544
• links to: GitHub Pull Request #608