[SUREFIRE-1516] Should surefire specialize test runner when test isolation (i.e., fork) is needed?

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.21.0
• Fix Version/s: 3.0.0-M5
• Component/s: Maven Surefire Plugin
• Assignee: Tibor Digana
• Reporter: Fork Script
• Created: 20/Apr/18 03:16
• Updated: 29/Dec/19 02:36
• Resolved: 29/Dec/19 02:33

Description

We observed that

 mvn test -DforkCount=1 -DreuseForks=false 

has high overhead for several projects we recently used.

We developed a specific runner for this option, which reduces test execution time by 49% on average. (Please see detailed results in the table below.)

Here is a link to our commit (see ForkStarter.java:388). Our approach is hackish, but we believe it can be engineered better if needed. Is this of interest to surefire? We would be happy to discuss this further and provide more results.

Detailed results are below. Machine we used: Intel i7-6700U CPU @ 3.40GHz with 16GB of RAM, running Ubuntu 17.10. We can share our script used to collect the numbers.

project	maven current (sec)	maven with our hack (sec)	time reduced by
activiti	1300.54	1069.20	17%
bukkit	18.98	4.80	74%
closurecompiler	399.42	271.68	31%
commonscodec	46.98	13.85	70%
commonsio	107.35	67.41	37%
commonslang	89.70	35.39	60%
commonsmath	391.20	110.79	71%
commonsnet	74.50	59.46	20%
commonsvalidator	37.30	13.24	64%
configuration	126.73	62.37	50%
crypto	23.75	7.14	69%
gedcom4j	195.43	64.10	67%
graphhopper	90.80	41.25	54%
jaxx	36.00	23.33	35%
la4j	15.23	7.60	50%
openfire	28.52	18.51	35%
retrofit	53.22	35.36	33%
scribejava	13.35	4.32	67%
vectorz	37.90	11.19	70%
ztexec	23.94	17.75	25%
Avg.	155.54	96.93	49%

Issue Links

• is related to: SUREFIRE-1658 TCP/IP Channel for forked Surefire JVM. Extensions API and SPI. Polymorphism for remote and local process communication.
• links to: GitHub Pull Request #253
• links to: GitHub Pull Request #259