Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1725

Surefire in JUnit Vintage mode distributes tests very unevenly between forks, causing poor parallelism

    XMLWordPrintableJSON

    Details

      Description

      To reproduce this issue, please run the attached test case using mvn clean verify

      You will see output similar to the following:

      [INFO]  T E S T S
      [INFO] -------------------------------------------------------
      [INFO] Running My7Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.017 s - in My7Test
      [INFO] Running My9Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.983 s - in My9Test
      [INFO] Running My4Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.998 s - in My4Test
      [INFO] Running My8Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.999 s - in My8Test
      [INFO] Running My6Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.001 s - in My6Test
      [INFO] Running My1Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1 s - in My1Test
      [INFO] Running My5Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.999 s - in My5Test
      [INFO] Running My3Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.997 s - in My3Test
      [INFO] Running My2Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1 s - in My2Test
      [INFO] Running My0Test
      Test in process 32569
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.999 s - in My0Test
      

      Notice how all 10 tests are being run on the same process. This is wrong because there should be 5 forks - and the tests should be evenly distributed between them.

      If Junit4 is used instead of Junit5+vintage (see the commented dependency in pom.xml) then this issue does not occur (probably because Surefire uses a different code path in that case).

      Please note that this issue was originally logged against Junit (https://github.com/junit-team/junit5/issues/2114) - but they suggested the problem lies within surefire.

      Thanks for any help!

        Attachments

        1. junit-5-vintage-bug.zip
          45 kB
          Gareth Daniel Smith

          Issue Links

            Activity

              People

              • Assignee:
                tibordigana Tibor Digana
                Reporter:
                garethdanielsmith Gareth Daniel Smith
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: