Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-865

surefire 2.12 with parallel=methods does not execute junit 4.7+ tests in parallel

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.12.1
    • Labels:
      None
    • Environment:
      Maven 2.2.1 and 3.0.4, JDK6, JUnit 4.10

      Description

      parallel=methods with junit does not appear to parallelize junit test execution in surefire 2.12 or 2.13-SNAPSHOT. parallel=classes works with 2.12, but parallel=methods only works for me in earlier versions of surefire like 2.10 and 2.11. I saw an earlier bug related to TestSuite classes but my tests are not using @Test. I have attached a very simple maven project that demonstrates this issue. It has 2 test classes with 2 tests each where each test calls Thread.sleep() for 4-8 seconds. Here are a few build commands to reproduce the bug:

      First run: mvn test -Dsurefire-version=2.11 -Dparallel=methods
      Notice that in 2.11 the tests execute in two threads and in parallel. Total execution time on my box was 15s.

      Then run: mvn test -Dsurefire-version=2.12 -Dparallel=methods
      Notice that in 2.12 the tests execute in the same main thread. Total execution time on my box was 26s.

      Finally run: mvn test -Dsurefire-version=2.12 -Dparallel=classes
      Notice that parallel=classes does correctly execute tests in two threads and in parallel. Total execution time on my box was 17s.

      Here is output from my machine when running these commands...

      Run:
      mvn test -Dsurefire-version=2.11 -Dparallel=methods

      OUTPUT:
      Concurrency config is parallel='methods', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
      Running com.overstock.BarTest
      Fri May 11 11:26:34 MDT 2012: BarTest calling sleep(5) in thread pool-1-thread-2
      Fri May 11 11:26:34 MDT 2012: BarTest calling sleep(4) in thread pool-1-thread-1
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.033 sec
      Running com.overstock.FooTest
      Fri May 11 11:26:39 MDT 2012: FooTest calling sleep(7) in thread pool-1-thread-3
      Fri May 11 11:26:39 MDT 2012: FooTest calling sleep(8) in thread pool-1-thread-4
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.008 sec

      Results :

      Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD SUCCESSFUL
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 15 seconds

      Run:
      mvn test -Dsurefire-version=2.12 -Dparallel=methods

      OUTPUT:
      Concurrency config is parallel='methods', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
      Running com.overstock.BarTest
      Fri May 11 11:25:53 MDT 2012: BarTest calling sleep(4) in thread main
      Fri May 11 11:25:58 MDT 2012: BarTest calling sleep(5) in thread main
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.017 sec
      Running com.overstock.FooTest
      Fri May 11 11:26:03 MDT 2012: FooTest calling sleep(7) in thread main
      Fri May 11 11:26:10 MDT 2012: FooTest calling sleep(8) in thread main
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.003 sec

      Results :

      Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD SUCCESSFUL
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 26 seconds

      Run:
      mvn test -Dsurefire-version=2.12 -Dparallel=classes

      OUTPUT:
      Concurrency config is parallel='classes', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
      Running com.overstock.BarTest
      Fri May 11 11:27:38 MDT 2012: BarTest calling sleep(4) in thread pool-1-thread-1
      Fri May 11 11:27:42 MDT 2012: BarTest calling sleep(5) in thread pool-1-thread-1
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.016 sec
      Running com.overstock.FooTest
      Fri May 11 11:27:38 MDT 2012: FooTest calling sleep(7) in thread pool-1-thread-2
      Fri May 11 11:27:45 MDT 2012: FooTest calling sleep(8) in thread pool-1-thread-2
      Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.016 sec

      Results :

      Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD SUCCESSFUL
      [INFO] ------------------------------------------------------------------------
      [INFO] Total time: 17 seconds

        Attachments

        1. surefire-parallel-junit.zip
          4 kB
          Neil Hartner

          Activity

            People

            • Assignee:
              krosenvold Kristian Rosenvold
              Reporter:
              nhartner Neil Hartner
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: