[SUREFIRE-2033] Regression: 3.0.0-M5 misidentifies JUnit 5 configuration as JUnit 4

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-M5
• Component/s: JUnit 5.x support
• Assignee: Unassigned
• Reporter: Robert James Oxspring
• Created: 03/Mar/22 09:45
• Updated: 02/Apr/22 19:58

Description

3.0.0-M5 misidentifies a clearly JUnit 5 configuration and attempts to run using JUnit 4.

In particular the attached project only has JUnit 5 dependencies:

<dependencies>
    <dependency>
        <groupId>org.junit.jupiter</groupId>
        <artifactId>junit-jupiter-engine</artifactId>
        <version>5.6.2</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.junit.platform</groupId>
        <artifactId>junit-platform-runner</artifactId>
        <version>1.6.2</version>
        <scope>test</scope>
    </dependency>
</dependencies>

and a JUnit 5 test:

package pkg;

import org.junit.jupiter.api.Test;

class JUnit5Test {
    @Test
    public void test() {

    }
}

When the project is run with with older version (as far back as 2.22.0) the test is found and run, but 3.0.0-M5 fails to run any test:

% mvn test -Dsurefire.version=2.22.0 | tail
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.337 s
[INFO] Finished at: 2022-03-03T09:42:27Z
[INFO] ------------------------------------------------------------------------ 

 

% mvn test -Dsurefire.version=3.0.0-M1 | tail
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.708 s
[INFO] Finished at: 2022-03-03T09:34:58Z
[INFO] ------------------------------------------------------------------------ 

% mvn test -Dsurefire.version=3.0.0-M2 | tail
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.701 s
[INFO] Finished at: 2022-03-03T09:36:26Z
[INFO] ------------------------------------------------------------------------

% mvn test -Dsurefire.version=3.0.0-M3 | tail
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.612 s
[INFO] Finished at: 2022-03-03T09:37:02Z
[INFO] ------------------------------------------------------------------------

% mvn test -Dsurefire.version=3.0.0-M4 | tail
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.619 s
[INFO] Finished at: 2022-03-03T09:37:37Z
[INFO] ------------------------------------------------------------------------

% mvn test -Dsurefire.version=3.0.0-M5 | tail
[INFO] Results:
[INFO] 
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.344 s
[INFO] Finished at: 2022-03-03T09:38:01Z
[INFO] ------------------------------------------------------------------------

Note the final run above using 3.0.0-M5 and logging: Tests run: 0

Issue Links

• is related to: SUREFIRE-1963 Detecting multiple test-frameworks
• relates to: SUREFIRE-1920 After adding junit-platform-runner dependency, Junit 5 test cases are ignored
• relates to: SUREFIRE-2036 Regression: 3.0.0-M5 fails with configured JUnit 5 provider