[SUREFIRE-2259] test-jar includes/excludes are not respected when running "mvn test"

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: Maven Surefire Plugin

Environment

Apache Maven 3.9.8 (36645f6c9b5079805ea5009217e36f2cffd34256)
Maven home: /home/famod/.sdkman/candidates/maven/current
Java version: 21.0.4, vendor: Eclipse Adoptium, runtime: /home/moldowan/.sdkman/candidates/java/21.0.4-tem
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.15.153.1-microsoft-standard-wsl2", arch: "amd64", family: "unix"

• Assignee: Unassigned
• Reporter: Falko Modler
• Created: 27/Aug/24 23:15
• Updated: 27/Aug/24 23:15

Description

Imagine a multi-module project consisting of two modules:

• module A provides a test-jar
• module B is running tests and depends on the test-jar of A

Module A builds the test-jar like this:

    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-jar-plugin</artifactId>
        <version>3.2.2</version>
        <executions>
          <execution>
            <goals>
              <goal>test-jar</goal>
            </goals>
            <configuration>
	          <includes>
	            <include>application.properties</include>
	            <include>org/acme/**</include>
	          </includes>
	    </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>

So it includes classes under org/acme and application.properties, but that module also has src/test/resources/application*-test*.properties that is not included in test-jar.

Now, if running tests in module B (that uses the test-jar of A), application-test.properties ist not on the test classpath when running (from the project root, with our without "clean"):

• mvn install
• mvn package
• mvn test -f B

This is all as expected.

But if just running "mvn test" the application-test.properties is suddenly on the classpath, which means that in that case includes/excludes are not respected.

I've attached a reproducer that I originally built for https://github.com/quarkusio/quarkus/issues/42580 which is also where this issue in surefire was spotted.
In that reproducer, module A is "core" and module B is "dist".
Relevant for this issue is SomeTest which fails only for "mvn test". SomeQuarkusTest is not relevant here.

I'm aware that "test" phase comes before the "package" phase but this is a nasty inconsistency nonetheless.