Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1851

NPE in SmartStackTraceParser causes false positive test results

    XMLWordPrintableJSON

    Details

      Description

      If throwing an exception within a test where the stack trace is null, a core utility (SmartStackTraceParser in comon-java5) will throw an NPE. This will cause the test suite to fail.

      What is especially scary about this is that it will not log that the test has failed, and in some configurations will even declare the build is successful and all tests passed. Additionally, the surefire report plugin will declare all tests passed too.

      An exception with a null stacktrace sounds odd, but is easy to do by mocking an exception with frameworks like Mockito. While people probably shouldn't be mocking exceptions, it definitely can and does happen.

      Not sure what versions exactly are affected. But it's affected the `common-java5` package since 2012. At least both JUnit 4 and 5 are affected, with surefire 2.22.1 or 3.0.0-M5. Logs attached.

      Can be reproduced with https://github.com/domdomegg/surefire-1851-demo

      I have a patch ready that I believe fixes this: https://github.com/apache/maven-surefire/pull/320

        Attachments

        1. junit5.7.0andsurefire3.0.0M5.txt
          10 kB
          Adam Jones
        2. junit4.11andsurefire2.22.2.txt
          6 kB
          Adam Jones

          Issue Links

            Activity

              People

              • Assignee:
                tibordigana Tibor Digana
                Reporter:
                domdomegg Adam Jones
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: