Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-869

Parallel test execution on class level runs not all tests but some twice

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.12
    • Fix Version/s: 2.12.1
    • Labels:
      None
    • Environment:

      Description

      When I run JUnit tests in parallel on the class level, Surefire behaves somehow erratically. For instance, having test classes TestA and TestB, Surefire sometimes runs TestA twice, TestB twice at other times - but never TestA and TestB in parallel.

      I attached a simple Maven project that helps reproduce this issue. Via mvn test -P sequential you can execute two test classes sequentially:

      ...
      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      Running mavendebug.TestAppFirstTest
      -----------------> TestAppFirstTest.test()
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.036 sec
      Running mavendebug.TestAppSecondTest
      -----------------> TestAppSecondTest.test()
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 sec
      
      Results :
      
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
      ...
      

      Running mvn test instead runs tests in parallel:

      ...
      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      
      -------------------------------------------------------
       T E S T S
      -------------------------------------------------------
      Concurrency config is parallel='classes', perCoreThreadCount=false, threadCount=4, useUnlimitedThreads=false
      Concurrency config is parallel='classes', perCoreThreadCount=false, threadCount=4, useUnlimitedThreads=false
      Running mavendebug.TestAppSecondTest
      Running mavendebug.TestAppSecondTest
      -----------------> TestAppSecondTest.test()
      -----------------> TestAppSecondTest.test()
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
      Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
      
      Results :
      
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
      ...
      

      As you can see, TestAppSecondTest is executed twice, while test TestAppFirstTest is not executed at all.

      Please note: I chose Blocker as priority as this caused (1) that our extensive test suite was not executed completely during the last weeks which we only discovered today, and (2) that caused tests that run against a database to fail with unexpected constraint violations just because those tests where executed twice in parallel.

        Attachments

        1. parallel-tests.tgz
          2 kB
          Martin Burger

          Activity

            People

            • Assignee:
              krosenvold Kristian Rosenvold
              Reporter:
              mburger Martin Burger
            • Votes:
              2 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: