Details
-
Bug
-
Status: Closed
-
Critical
-
Resolution: Fixed
-
2.20, 2.20.1, 2.21.0, 2.22.0, 2.22.1, 2.22.2, 3.0.0-M1, 3.0.0-M2, 3.0.0-M3, 3.0.0-M4, 3.0.0-M5
-
None
-
Have reproduced with both Java 8 and 11:
openjdk version "1.8.0_292"
openjdk version "11.0.11" 2021-04-20
and maven
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Description
Upon upgrading our maven-surefire-plugin from version 2.15 to version 2.21.0. We noticed that the number of tests being run by our jenkins test job changed. Upon further investigation we found a test class with 7 test cases that were not being reported at all (not as passed, failed, or skipped).
When we ran just that test class from the command line it failed with
[ERROR] Crashed tests:
[ERROR] com.sensus.dataextraction.reportcomparetool.TestFlowPreHHFReports
[ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
[ERROR] Command was cmd.exe /X /C "C:\tools\java8\jre\bin\java org.apache.maven.surefire.booter.ForkedBooter C:\tools\cygwin\tmp\surefire7735045605128719483 2021-09-17T10-56-30_089-jvmRun1 surefire4447346130857727490tmp surefire_04489252332082939918tmp"
[ERROR] Process Exit Code: 0
[ERROR] Crashed tests:
but did not produce any output in the target/surefire-reports directory explaining why the failures were not reported in the jobs test report. Trying different maven-surefire-plugin versions the tests pass successfully with version 2.19.1, but fail with every version tried between 2.20 and 3.0.0-M5. Note we run our tests with these options:
<reuseForks>false</reuseForks>
<forkCount>1</forkCount>
<reportFormat>xml</reportFormat>
I also noticed that the tests in question did produce a lot of logging output. I found that the tests would pass if I added the configuration option:
<redirectTestOutputToFile>true</redirectTestOutputToFile>
I have reproduced the problem with a simpler test module that is attached. Upon playing with the output in this module the issue appears to happen when the output is about 1.2MB or higher.
Not sure what changed between version 2.19.1 and 2.20 with how output from the forked jvm is collected but it appear to have a problem if there is too much output. If the tests were being reported as failed then this issue would not be a huge problem, but having tests just not be reported because their output grows is a critical issue for us as we may not notice that the test has effectively been dropped from our test suite, unless we closely audit the ~30000 test cases we run every run to detect missing tests which is hard to do as new tests are constantly being added.
Attachments
Attachments
Issue Links
- is duplicated by
-
-
- Closed
-
