[SUREFIRE-1492] Failsafe doesn't fail when a TestEngine throws an Error

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.19.1
• Fix Version/s: waiting-for-feedback
• Component/s: classloading, JUnit 5.x support
• Assignee: Unassigned
• Reporter: Mateusz Stefek
• Created: 01/Mar/18 11:40
• Updated: 26/Mar/22 23:42

Description

My Jupiter engine fails due to some classloader hell. The logs are below. But that's not the point.

In such case, maven-failsafe-plugin should fail, but it exits as if nothing happened. Even with  failIfNoTests>true</failIfNoTests> the plugin finishes and generates failsafe-summary.xml with <completed>0</completed>

Forking command line: cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_91\jre\bin\java" -jar C:\Users\matste\uptime\uptime\ui\ui-demo\target\surefire\surefirebooter5276177102867119205.jar C:\Users\matste\uptime\uptime\ui\ui-demo\target\surefire\surefire268158133198646317tmp C:\Users\matste\uptime\uptime\ui\ui-demo\target\surefire\surefire_05629521514361875618tmp"
Mar 01, 2018 12:29:15 PM org.junit.platform.launcher.core.DefaultLauncher handleThrowable
WARNING: TestEngine with ID 'junit-jupiter' failed to discover tests
java.lang.BootstrapMethodError: java.lang.NoSuchMethodError: org.junit.platform.commons.util.ReflectionUtils.isNotPrivate(Ljava/lang/reflect/Member;)Z
at org.junit.jupiter.engine.discovery.JavaElementsResolver.resolveContainedMethods(JavaElementsResolver.java:189)
at org.junit.jupiter.engine.discovery.JavaElementsResolver.resolveChildren(JavaElementsResolver.java:177)
at java.lang.Iterable.forEach(Iterable.java:75)
at org.junit.jupiter.engine.discovery.JavaElementsResolver.resolveClass(JavaElementsResolver.java:61)
at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.lambda$resolve$3(DiscoverySelectorResolver.java:69)
at java.util.ArrayList.forEach(ArrayList.java:1249)
at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolve(DiscoverySelectorResolver.java:68)
at org.junit.jupiter.engine.discovery.DiscoverySelectorResolver.resolveSelectors(DiscoverySelectorResolver.java:50)
at org.junit.jupiter.engine.JupiterTestEngine.discover(JupiterTestEngine.java:61)
at org.junit.platform.launcher.core.DefaultLauncher.discoverEngineRoot(DefaultLauncher.java:130)
at org.junit.platform.launcher.core.DefaultLauncher.discoverRoot(DefaultLauncher.java:117)
at org.junit.platform.launcher.core.DefaultLauncher.discover(DefaultLauncher.java:82)
at org.junit.platform.surefire.provider.TestPlanScannerFilter.accept(TestPlanScannerFilter.java:50)
at org.apache.maven.surefire.util.DefaultScanResult.applyFilter(DefaultScanResult.java:98)
at org.junit.platform.surefire.provider.JUnitPlatformProvider.scanClasspath(JUnitPlatformProvider.java:121)
at org.junit.platform.surefire.provider.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:111)
at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
Caused by: java.lang.NoSuchMethodError: org.junit.platform.commons.util.ReflectionUtils.isNotPrivate(Ljava/lang/reflect/Member;)Z
at java.lang.invoke.MethodHandleNatives.resolve(Native Method)
at java.lang.invoke.MemberName$Factory.resolve(MemberName.java:977)
at java.lang.invoke.MemberName$Factory.resolveOrFail(MemberName.java:1002)
at java.lang.invoke.MethodHandles$Lookup.resolveOrFail(MethodHandles.java:1390)
at java.lang.invoke.MethodHandles$Lookup.linkMethodHandleConstant(MethodHandles.java:1746)
at java.lang.invoke.MethodHandleNatives.linkMethodHandleConstant(MethodHandleNatives.java:477)
... 19 more

Results :

Tests run: 0, Failures: 0, Errors: 0, Skipped: 0