Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1433

rerunFailingTestsCount does not help with crashes

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.20.1
    • Fix Version/s: None
    • Component/s: process forking
    • Labels:
      None
    • Environment:
      Linux, Java 8u131, Maven 3.5.0

      Description

      I have a CI job running lots of very slow tests, and am using -Dsurefire.rerunFailingTestsCount=2 -Dmaven.test.failure.ignore -Dsurefire.timeout=7200 -DreuseForks=false -DforkCount=2 to run Maven. It will typically run for about three hours, collecting a lot of passing tests, a few flaking tests with subsequent passes, and a few true failures. But then it will abruptly halt with the infamous

      Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project ...: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
      

      The timing is unpredictable but the build seems to usually fail this way; sometimes I get lucky and it runs through all the tests, taking a number of hours. Diagnosing the crash is not likely to be straightforward, and frankly I do not care much—I just want Surefire to try again. Yet the rerunFailingTestsCount flag does not accomplish this—it only retries normal failures.

      You can see this from running the attached project. While you might expect it to go through 100 runs of the test, some failing politely and some crashing, instead you see something like

      ...
      --- maven-surefire-plugin:2.20.1:test (default-test) @ demo ---
      
      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      Running demo.CrashingTest
      failing
      failing
      failing
      crashing
      
      Results:
      
      Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
      
      ------------------------------------------------------------------------
      BUILD FAILURE
      ------------------------------------------------------------------------
      Total time: 2.456 s
      Finished at: 2017-10-07T15:11:04-04:00
      Final Memory: 11M/222M
      ------------------------------------------------------------------------
      Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20.1:test (default-test) on project demo: There are test failures.
      ...
      

        Attachments

        1. demo.zip
          1 kB
          Jesse Glick

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              jglick@netbeans.org Jesse Glick
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: