[SUREFIRE-913] forkMode always - RejectedExecutionException when > 500 tests

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.12.4
• Fix Version/s: 2.13
• Component/s: Maven Surefire Plugin
• Flags: Patch
• Assignee: Kristian Rosenvold
• Reporter: Gary Tully
• Created: 09/Oct/12 05:33
• Updated: 29/Jun/15 02:45
• Resolved: 08/Nov/12 11:19

Description

executing more than 500 tests using forkMode=always gives a RejectedExecutionException because the executor has a bounded work queue with a hardcoded limit of 500.

From the activemq jenkins ci build, the error is reported when 500 tests have run, but it occurs when they are initially scheduled by org.apache.maven.plugin.surefire.booterclient.ForkStarter#runSuitesForkPerTestSet

Waiting for Jenkins to finish collecting data
mavenExecutionResult exceptions not empty
message : Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project activemq-core: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test failed.
cause : Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test failed.
Stack trace : 
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project activemq-core: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test failed.
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:225)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:319)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
	at org.jvnet.hudson.maven3.launcher.Maven3Launcher.main(Maven3Launcher.java:79)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239)
	at org.jvnet.hudson.maven3.agent.Maven3Main.launch(Maven3Main.java:158)
	at hudson.maven.Maven3Builder.call(Maven3Builder.java:98)
	at hudson.maven.Maven3Builder.call(Maven3Builder.java:64)
	at hudson.remoting.UserRequest.perform(UserRequest.java:118)
	at hudson.remoting.UserRequest.perform(UserRequest.java:48)
	at hudson.remoting.Request$2.run(Request.java:326)
	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test failed.
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:110)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
	... 27 more
Caused by: java.util.concurrent.RejectedExecutionException
	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:658)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:92)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:168)
	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:121)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:740)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAllProviders(AbstractSurefireMojo.java:682)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:648)
	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:586)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
	... 28 more

Full output: https://builds.apache.org/job/ActiveMQ/1096/consoleText

A fix is to make the executor unbounded, which works with the activemq test suite, letting all tests queue up for execution for the single core thread to work through:

@@ -142,8 +144,8 @@ public class ForkStarter
     {
 
         ArrayList<Future<RunResult>> results = new ArrayList<Future<RunResult>>( 500 );
-        ExecutorService executorService = new ThreadPoolExecutor( forkCount, forkCount, 60, TimeUnit.SECONDS,
-                                                                  new ArrayBlockingQueue<Runnable>( 500 ) );
+        ExecutorService executorService = new ThreadPoolExecutor( forkCount, forkCount, 5, TimeUnit.SECONDS,
+                                                                  new LinkedBlockingQueue<Runnable>( ) );

Issue Links

• relates to: AMBARI-12196 Ambari-server component build fails with 'maven-surefire-plugin' failing with more than 500 tests
• relates to: SUREFIRE-799 Allow test parallelisation when forkMode=always