[SUREFIRE-1935] Upgrade to JUnit Platform 1.8, start Launcher via LauncherSession

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-M5
• Fix Version/s: 3.0.0-M6
• Component/s: JUnit 5.x support, process forking
• Assignee: Tibor Digana
• Reporter: Emond Papegaaij
• Created: 13/Aug/21 10:53
• Updated: 20/Jan/22 08:34
• Resolved: 20/Jan/22 08:33

Description

When running tests with forkCount > 1 on the JUnit 5 Platform, the JUnitPlatformProvider keeps restarting the entire launcher, causing the life cycle of the tests to change. When an Extension registers an expensive resource in the root store, this resource is now created over and over again for every test class. An example of such an Extension is the ArquillianExtension. The recreation of this resource causes the tests to create and destroy all containers associated with the test for every class, rather than once per fork.

I've attached a very simple example project that contains 4 tests and a dummy extension that registers an object in the root store. The output clearly indicates the problem:

$ mvn surefire:test -DforkCount=1
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------------< nl.topicus:forkedtests >-----------------------
[INFO] Building forkedtests 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running nl.topicus.test.Test2
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.021 s - in nl.topicus.test.Test2
[INFO] Running nl.topicus.test.Test1
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001 s - in nl.topicus.test.Test1
[INFO] Running nl.topicus.test.Test3
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s - in nl.topicus.test.Test3
[INFO] Running nl.topicus.test.Test4
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.001 s - in nl.topicus.test.Test4
Discarding heavy calculation
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  8.816 s
[INFO] Finished at: 2021-08-13T12:50:40+02:00
[INFO] ------------------------------------------------------------------------

Compared to running with forkCount=2 (notice how the heavy calculation is performed 4 times, where it should only run twice):

$ mvn surefire:test -DforkCount=2
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------------< nl.topicus:forkedtests >-----------------------
[INFO] Building forkedtests 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-surefire-plugin:3.0.0-M5:test (default-cli) @ forkedtests ---
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running nl.topicus.test.Test2
[INFO] Running nl.topicus.test.Test1
Performing heavy calculation
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.025 s - in nl.topicus.test.Test2
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.022 s - in nl.topicus.test.Test1
Discarding heavy calculation
Discarding heavy calculation
[INFO] Running nl.topicus.test.Test3
[INFO] Running nl.topicus.test.Test4
Performing heavy calculation
Performing heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s - in nl.topicus.test.Test3
Discarding heavy calculation
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.002 s - in nl.topicus.test.Test4
Discarding heavy calculation
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.893 s
[INFO] Finished at: 2021-08-13T12:50:48+02:00
[INFO] ------------------------------------------------------------------------

Issue Links

• links to: GitHub Pull Request #389