Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1741

Exceptions in parameterized test sources are ignored

    XMLWordPrintableJSON

    Details

      Description

      Associated versions on my end at time of testing:

      • JUnit Jupiter 5.5.1
      • JDK 11.0.5
      • Maven 3.6.1

       

      If the following code is run, either for Surefire or Failsafe, only two of the test methods are logged as having been detected at all. The third one is completely ignored as if it didn't exist.

      public class ExampleTest {
      
        @Test
        public void shouldRunAndPassAsExpected() {
        }
      
        public static Stream<Arguments> nothing() {
          return Stream.of(Arguments.arguments());
        }
      
        @ParameterizedTest
        @MethodSource("nothing")
        public void shouldRunAndFailAsExpected() {
          Assertions.fail();
        }
      
        static Stream<Arguments> throwException() {
          if (true) throw new RuntimeException("Stop");
          return Stream.of(Arguments.arguments());
        }
      
        @ParameterizedTest
        @MethodSource("throwException")
        void willNotBeLoggedAsFailingDespiteNotRunning() {
          Assertions.fail();
        }
      }
      

      This is particularly obvious if the first two test methods are commented out, as running mvn test will log that 0 tests were run.

       

      In case it helps, here's the output I'm seeing when running it with the first two tests commented out:

      [INFO] -------------------------------------------------------
      [INFO]  T E S T S
      [INFO] -------------------------------------------------------
      [INFO] Running ExampleIT
      [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 s - in ExampleIT
      [INFO] 
      [INFO] Results:
      [INFO] 
      [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                tibordigana Tibor Digana
                Reporter:
                ajohnston Albert Johnston
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h