Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1127

Failsafe project does not fail in verify phase when a test case object errors during initialization

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.18
    • Fix Version/s: 2.19
    • Component/s: Maven Failsafe Plugin
    • Labels:
      None
    • Environment:
      JDK 1.8.0_05 on Windows 7 used to recreate

      Description

      Sample project attached.

      When running a "mvn verify" - the attached project will succeed, even though there is an integration test that should obviously fail.

      The integration test case does not initialize correctly (it will throw a null pointer exception in one of its object initializers), which should be a test failure and therefore a project failure.

      I've traced this down to the fact that the generated failsafe-summary.xml file contains an empty failureMessage element:

      <?xml version="1.0" encoding="UTF-8"?>
      <failsafe-summary result="254" timeout="false">
        <completed>0</completed>
        <errors>0</errors>
        <failures>0</failures>
        <skipped>0</skipped>
        <failureMessage></failureMessage>
      </failsafe-summary>
      

      And so the failsafe verifier doesn't consider the project a failure. The message seems to be blank because when the failsafe plugin is writing the results, it uses the org.apache.maven.plugins.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked method to write out the first forked exception, which then calls org.apache.maven.surefire.suite.RunResult.failure() method, which then calls getStackTrace() on the exception.

      The getStackTrace() method looks like this:

          private static String getStackTrace( Exception e )
          {
              if ( e == null )
              {
                  return null;
              }
              ByteArrayOutputStream out = new ByteArrayOutputStream();
              PrintWriter pw = new PrintWriter( out );
              e.printStackTrace( pw );
              return new String( out.toByteArray() );
          }
      

      And it specifically is doing a toBytearray on the output stream without flushing or closing the printwriter first. At least with my JVM 1.8.0_05 version, this seems to be a problem as the printwriter still maintains the trace description in a buffer.

      When I put a breakpoint on this and specifically flush or close the printwriter before the toByteArray, the project fails as expected.

      I'm not sure why this is failsafe specific, but a surefire version test case (also included in the attachment project) fails appropriately.

      I think a simple patch is to update RunResult.java to close the printwriter before getting the resulting stacktrace string. i.e.

          private static String getStackTrace( Exception e )
          {
              if ( e == null )
              {
                  return null;
              }
              ByteArrayOutputStream out = new ByteArrayOutputStream();
              PrintWriter pw = new PrintWriter( out );
              try {
                  e.printStackTrace( pw );
              } finally {
                  pw.close();
              }
              return new String( out.toByteArray() );
          }
      

        Attachments

        1. failsafe-test.zip
          3 kB
          Bret Goldsmith

          Issue Links

            Activity

              People

              • Assignee:
                tibordigana Tibor Digana
                Reporter:
                bretg Bret Goldsmith
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: