Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.0.0-M6
-
None
-
None
Description
We are running Maven Surefire plug-in in a larger project separately from the main build due to scalability reasons. This may lead top situation when the main build (compile) works successfully and an incomplete test classpath goes unnoticed. In such a case the Maven Surefire plug-in fails completely. Our wish is to never fail the build but report any issues as test failures.
We are using:
<failIfNoTests>false</failIfNoTests> <failIfNoSpecifiedTests>false</failIfNoSpecifiedTests> <testFailureIgnore>true</testFailureIgnore>
Here is an example of a classpath issue:
Caused by: org.apache.maven.surefire.api.util.SurefireReflectionException: org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-vintage' failed to discover tests at org.apache.maven.surefire.api.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:173) at org.apache.maven.surefire.api.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:76) at org.apache.maven.surefire.api.util.ReflectionUtils.invokeGetter(ReflectionUtils.java:70) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.getSuites(ProviderFactory.java:145) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.getSuitesIterator(ForkStarter.java:751) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkOnceMultiple(ForkStarter.java:351) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:326) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:269) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1332) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:1165) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:929) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:140) ... 11 common frames omitted Caused by: org.junit.platform.commons.JUnitException: TestEngine with ID 'junit-vintage' failed to discover tests at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:160) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverSafely(EngineDiscoveryOrchestrator.java:134) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:108) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discover(EngineDiscoveryOrchestrator.java:80) at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:110) at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:78) at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.discover(DefaultLauncherSession.java:81) at org.apache.maven.surefire.junitplatform.LazyLauncher.discover(LazyLauncher.java:48) at org.apache.maven.surefire.junitplatform.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:56) at org.apache.maven.surefire.api.util.DefaultScanResult.applyFilter(DefaultScanResult.java:102) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:167) at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.getSuites(JUnitPlatformProvider.java:109) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.maven.surefire.api.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:165) ... 22 common frames omitted Caused by: java.lang.NoClassDefFoundError: org/custommonkey/xmlunit/DifferenceListener at java.base/java.lang.Class.getDeclaredMethods0(Native Method) at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166) at java.base/java.lang.Class.privateGetPublicMethods(Class.java:3191) at java.base/java.lang.Class.getMethods(Class.java:1904) at org.junit.platform.commons.util.ReflectionUtils.getDefaultMethods(ReflectionUtils.java:1518) at org.junit.platform.commons.util.ReflectionUtils.getDeclaredMethods(ReflectionUtils.java:1491) at org.junit.platform.commons.util.ReflectionUtils.findAllMethodsInHierarchy(ReflectionUtils.java:1433) at org.junit.platform.commons.util.ReflectionUtils.findMethods(ReflectionUtils.java:1417) at org.junit.platform.commons.util.ReflectionUtils.findMethods(ReflectionUtils.java:1403) at org.junit.vintage.engine.descriptor.TestSourceProvider.lambda$findMethod$1(TestSourceProvider.java:75) at java.base/java.util.HashMap.computeIfAbsent(HashMap.java:1134) at java.base/java.util.Collections$SynchronizedMap.computeIfAbsent(Collections.java:2682) at org.junit.vintage.engine.descriptor.TestSourceProvider.findMethod(TestSourceProvider.java:75) at org.junit.vintage.engine.descriptor.TestSourceProvider.computeTestSource(TestSourceProvider.java:56) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) at org.junit.vintage.engine.descriptor.TestSourceProvider.findTestSource(TestSourceProvider.java:47) at org.junit.vintage.engine.discovery.RunnerTestDescriptorPostProcessor.addChildrenRecursively(RunnerTestDescriptorPostProcessor.java:62) at org.junit.vintage.engine.discovery.RunnerTestDescriptorPostProcessor.applyFiltersAndCreateDescendants(RunnerTestDescriptorPostProcessor.java:41) at org.junit.vintage.engine.discovery.VintageDiscoverer.discover(VintageDiscoverer.java:46) at org.junit.vintage.engine.VintageTestEngine.discover(VintageTestEngine.java:64) at org.junit.platform.launcher.core.EngineDiscoveryOrchestrator.discoverEngineRoot(EngineDiscoveryOrchestrator.java:152) ... 38 common frames omitted Caused by: java.lang.ClassNotFoundException: org.custommonkey.xmlunit.DifferenceListener at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:476) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522) at org.apache.maven.surefire.booter.IsolatedClassLoader.loadClass(IsolatedClassLoader.java:100) ... 59 common frames omitted
I reported the issue also at https://github.com/junit-team/junit5/issues/2971 but I wonder if this can be handled by Maven Surefire directly independent of the test engine.