[SUREFIRE-1676] surefire-junit47 is not running JUnit4 tests

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 3.0.0-M3
• Component/s: Maven Surefire Plugin
• Environment: Windows 10 and Centos 7 behave the same. ; java 1.8.161
• Assignee: Tibor Digana
• Reporter: Bobby Erwin
• Created: 10/Jul/19 17:31
• Updated: 18/Jul/19 20:00
• Resolved: 18/Jul/19 20:00

Description

In a maven project with lots of JUnit 4 tests, surefire-junit47 is not executing the tests.

There are no testng tests in this project, and there is no testng in the pom. But this project has a dependency on another project that does have testng in the pom. You can see it import testng in the mvn -X output, attached.

With testng on the classpath, surefire tries to execute tests through testng. So we're configuring our test execution based on this page: https://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html

For reference sake, here is the documentation I'm working from:https://maven.apache.org/surefire/maven-surefire-plugin/examples/inclusion-exclusion.html

Curiously enough, it appears to find the test classes, but doesn't execute the methods. That is, if processes all the static fields and @Parameterized methods, but it doesn't execute any @Test methods.

Here are a couple of pom configurations that show the problem.

Given this test class organization:

- src/main/test/
   - com.mycomp.qc.core.account
     - CopyAccountTests.java
     - CreateAccountTests.java
     - DeleteAccountTests.java
     - ListAccountTests.java
     - ReadAccountTests.java
     - UpdateAccountTests.java
   - com.mycomp.qc.core.product
     - CopyProductTests.java
     - CreateProductTests.java
     - DeleteProductTests.java
     - ListProductTests.java
     - ReadProductTests.java
     - UpdateProductTests.java
   - ..... and 300 more packages .....

 

And given this test class structure:

package com.mycomp.qc.core.account;

import org.junit.Assert;
import org.junit.Test;
.... and more ....

public class CopyAccountTests {

    @Test
    public void copyAccount1() {
        Assert.assertTrue("pass", true);
    }

    @Test
    public void copyAccount2() {
        Assert.assertTrue("fail", false);
    }

.... and more ....
}

 

pom config 1: Specifically include Account tests, by pattern
Runs all the Account tests, just as the documentation indicates.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
        <includes>
            <include>*Account*</include>
        </includes>
        <threadCount>1</threadCount>
    </configuration>
    <executions>
        <execution>
            <id>default-test</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

pom config 2: Specifically include Account tests, by pattern

Runs all the Account and Product tests, just as the documentation indicates.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
        <includes>
            <include>*Account*</include>
            <include>*Product*</include>
        </includes>
        <threadCount>1</threadCount>
    </configuration>
    <executions>
        <execution>
            <id>default-test</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

pom config 3: Include all tests, based on default surefire

Finds and initializes test classes, but does not execute any @Test methods.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
        <threadCount>1</threadCount>
    </configuration>
    <executions>
        <execution>
            <id>default-test</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

pom config 4: Include all tests, by pattern

Finds and initializes test classes, but does not execute any @Test methods.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>${surefire.version}</version>
    <dependencies>
        <dependency>
            <groupId>org.apache.maven.surefire</groupId>
            <artifactId>surefire-junit47</artifactId>
            <version>${surefire.version}</version>
        </dependency>
    </dependencies>
    <configuration>
        <testFailureIgnore>true</testFailureIgnore>
        <includes>
            <include>*Test*</include>
        </includes>
        <threadCount>1</threadCount>
    </configuration>
    <executions>
        <execution>
            <id>default-test</id>
            <phase>test</phase>
            <goals>
                <goal>test</goal>
            </goals>
        </execution>
    </executions>
</plugin>

What I've tried:

1. Obviously, from the examples, I've tried different include patterns in the pom. See the results outlined above.

1. Configured a new project, with all the same imports and just a few small tests. All the include patterns above behaved as outlined in the documentation.

1. Switched the surefire provider to surefire-junit4. This, in fact, executed all tests, but we ran into other problems.

1. Ran mvn -X, mainly to look for testng problems, based on this answer: Surefire is not picking up Junit 4 tests.

1. mvn -X showed that the default maven-resources-plugin pulls in junit 3.8.x, which the doc says might cause problems. Updated resources to 3.1.0, but it didn't fix my problem.

mvn -X output
attached