[SUREFIRE-2189] Tests in src/main/java are not detected

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.1.2
• Component/s: JUnit 5.x support, Maven Surefire Plugin
• Environment: Java 17, with or without JPMS
• Assignee: Unassigned
• Reporter: Robert Važan
• Created: 01/Aug/23 09:16
• Updated: 01/Aug/23 09:31

Description

There are sometimes reasons to put JUnit tests under src/main/java next to production code. Surefire cannot detect these tests and runs only tests under src/test/java.

I tried to set testClassesDirectory to target/classes. That causes surefire to run tests under src/main/java, but then tests under src/test/java are ignored. Changing testClassesDirectory also causes weird module loading exceptions in a bigger project with more dependencies.

If you are curious why anyone would put tests in production code, here are my reasons:

1. I want to run all tests in the background after the app starts in dev environment. Since the tests rarely fail, this lets me shorten edit-test-run cycle into edit-run cycle.
2. I want to visually annotate app UI that has associated tests.
3. I want to run tests in production environment as a sanity check.
4. I want to check compatibility between the app and its environment. This could be done with ordinary code, but structuring it as JUnit tests is convenient and results in clean code. I want to run these tests via surefire too to make sure they aren't already failing in dev environment.
5. Keeping tests under src/main/java makes them more salient during development. Tests hidden under src/test/java tend to be forgotten during refactoring or when the tested code changes.