[SUREFIRE-2007] Java 17 TestEngine with ID 'junit-vintage' failed to discover tests

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 3.0.0-M5
• Component/s: classloading, Junit 4.x support, Maven Surefire Plugin

Environment

Apache Maven 3.8.1 (05c21c65bdfed0f71a2f2ada8b84da59348c4c5d)
Maven home: /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/MVN3
Java version: 17.0.1, vendor: Eclipse Adoptium, runtime: /home/jenkins/tools/hudson.model.JDK/JDK17/jdk-17.0.1+12
Default locale: en_GB, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-74-generic", arch: "amd64", family: "unix"

• Assignee: Tibor Digana
• Reporter: Ben Middleton
• Created: 11/Feb/22 10:10
• Updated: 22/Feb/22 15:26
• Resolved: 22/Feb/22 13:10

Description

We've recently switch our builds from Java 8 to Java 17, and we see some Failsafe tests failing with messages such as these:

org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-vintage' failed to discover tests
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160)
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:134)
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:108)
        at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:80)
        at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:110)
        at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:78)
        at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.discover(DefaultLauncherSession.java:81)
        at org.junit.platform.launcher.core.SessionPerRequestLauncher.discover(SessionPerRequestLauncher.java:46)
        at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:56)
        at org.apache.maven.surefire.api.util.DefaultScanResult.applyFilter(DefaultScanResult.java:102)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:147)
        at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:128)
        at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
        at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
        at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
        at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
Caused by: java.lang.NoClassDefFoundError: com/mycompany/broker/StorageException
        at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
        at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3402)
        at java.base/java.lang.Class.privateGetPublicMethods(Class.java:3427)
        at java.base/java.lang.Class.getMethods(Class.java:2019)
        at org.junit.platform.commons.util.ReflectionUtils.getDefaultMethods(ReflectionUtils.java:1518)
        at org.junit.platform.commons.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:1491)
        at org.junit.platform.commons.util.ReflectionUtils.findAllMethodsInHierarchy(ReflectionUtils.java:1433)
        at org.junit.platform.commons.util.ReflectionUtils.findMethods(ReflectionUtils.java:1417)
        at org.junit.platform.commons.util.ReflectionUtils.findMethods(ReflectionUtils.java:1403)
        at org.junit.vintage.engine.descriptor.TestSourceProvider.lambda$findMethod$1(TestSourceProvider.java:75)
        at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1220)
        at java.base/java.util.Collections$SynchronizedMap.computeIfAbsent(Collections.java:2760)
        at org.junit.vintage.engine.descriptor.TestSourceProvider.findMethod(TestSourceProvider.java:75)
        at org.junit.vintage.engine.descriptor.TestSourceProvider.computeTestSource(TestSourceProvider.java:56)
        at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1708)
        at org.junit.vintage.engine.descriptor.TestSourceProvider.findTestSource(TestSourceProvider.java:47)
        at org.junit.vintage.engine.discovery.RunnerTestDescriptorPostProcessor.addChildrenRecursively(RunnerTestDescriptorPostProcessor.java:62)
        at org.junit.vintage.engine.discovery.RunnerTestDescriptorPostProcessor.applyFiltersAndCreateDescendants(RunnerTestDescriptorPostProcessor.java:41)
        at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:46) 

However, the class in question does exist in the classpath. We've tried updating the tests to the latest JUnit 4 and 5 versions, but we see the same issue on each.

However, it I build the latest master branch of maven-surefire-plugin locally, and switch to this M6-SNAPSHOT, then the classpath loading works correctly.

When will a new release of the plugin be pushed? Do you know which change since M5 fixed this class loading issue on Java 17?

Thanks