[SUREFIRE-865] surefire 2.12 with parallel=methods does not execute junit 4.7+ tests in parallel

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.12.1
• Component/s: Junit 4.7+ (parallel) support
• Environment: Maven 2.2.1 and 3.0.4, JDK6, JUnit 4.10
• Assignee: Kristian Rosenvold
• Reporter: Neil Hartner
• Created: 11/May/12 13:21
• Updated: 08/Jul/12 15:17
• Resolved: 08/Jul/12 15:17

Description

parallel=methods with junit does not appear to parallelize junit test execution in surefire 2.12 or 2.13-SNAPSHOT. parallel=classes works with 2.12, but parallel=methods only works for me in earlier versions of surefire like 2.10 and 2.11. I saw an earlier bug related to TestSuite classes but my tests are not using @Test. I have attached a very simple maven project that demonstrates this issue. It has 2 test classes with 2 tests each where each test calls Thread.sleep() for 4-8 seconds. Here are a few build commands to reproduce the bug:

First run: mvn test -Dsurefire-version=2.11 -Dparallel=methods
Notice that in 2.11 the tests execute in two threads and in parallel. Total execution time on my box was 15s.

Then run: mvn test -Dsurefire-version=2.12 -Dparallel=methods
Notice that in 2.12 the tests execute in the same main thread. Total execution time on my box was 26s.

Finally run: mvn test -Dsurefire-version=2.12 -Dparallel=classes
Notice that parallel=classes does correctly execute tests in two threads and in parallel. Total execution time on my box was 17s.

—

Here is output from my machine when running these commands...

Run:
mvn test -Dsurefire-version=2.11 -Dparallel=methods

OUTPUT:
Concurrency config is parallel='methods', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
Running com.overstock.BarTest
Fri May 11 11:26:34 MDT 2012: BarTest calling sleep(5) in thread pool-1-thread-2
Fri May 11 11:26:34 MDT 2012: BarTest calling sleep(4) in thread pool-1-thread-1
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.033 sec
Running com.overstock.FooTest
Fri May 11 11:26:39 MDT 2012: FooTest calling sleep(7) in thread pool-1-thread-3
Fri May 11 11:26:39 MDT 2012: FooTest calling sleep(8) in thread pool-1-thread-4
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.008 sec

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 15 seconds

Run:
mvn test -Dsurefire-version=2.12 -Dparallel=methods

OUTPUT:
Concurrency config is parallel='methods', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
Running com.overstock.BarTest
Fri May 11 11:25:53 MDT 2012: BarTest calling sleep(4) in thread main
Fri May 11 11:25:58 MDT 2012: BarTest calling sleep(5) in thread main
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.017 sec
Running com.overstock.FooTest
Fri May 11 11:26:03 MDT 2012: FooTest calling sleep(7) in thread main
Fri May 11 11:26:10 MDT 2012: FooTest calling sleep(8) in thread main
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.003 sec

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26 seconds

Run:
mvn test -Dsurefire-version=2.12 -Dparallel=classes

OUTPUT:
Concurrency config is parallel='classes', perCoreThreadCount=true, threadCount=2, useUnlimitedThreads=false
Running com.overstock.BarTest
Fri May 11 11:27:38 MDT 2012: BarTest calling sleep(4) in thread pool-1-thread-1
Fri May 11 11:27:42 MDT 2012: BarTest calling sleep(5) in thread pool-1-thread-1
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.016 sec
Running com.overstock.FooTest
Fri May 11 11:27:38 MDT 2012: FooTest calling sleep(7) in thread pool-1-thread-2
Fri May 11 11:27:45 MDT 2012: FooTest calling sleep(8) in thread pool-1-thread-2
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.016 sec

Results :

Tests run: 4, Failures: 0, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17 seconds