[SUREFIRE-726] Test list preprocessor support for tests to be run

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.9
• Flags: Patch
• Assignee: Unassigned
• Reporter: Janmejay Singh
• Created: 17/Apr/11 20:39
• Updated: 30/May/23 18:11

Description

This exposes an interface(named TestListPreprocessor, which has a method named preprocessTestClasses) that can be implemented by a user and injected into surefire plugin configuration to have it used for preprocessing list of tests to be run. Given the original test list, user's preprocessing algorithm can choose to prune or reorder the list and return it back from TestListPreprocessor#preprocessTestClasses, which is then used as the list of tests to be executed.

The patches attached expose a configuration element named "testPreprocessor" the default value of which is "none". User can choose to set it to a value matching the format "<fully-qualified-class-name>[<group-id>:<artifact-id>:<version>]" and have the class loaded and called with list of tests to be run. The list of tests returned by the call is then considered for execution.

Patch description:
The feature is done in 2 patches. Description follows:

0001-Adds-test-list-preprocessor-support-which-loads-an-o.patch
This patch actually adds the feature and unit tests. It adds the interface, an abstraction that encapsulates aforementioned configuration, booter serialization/deserialization, directory-scanner changes etc. The changes have effect in both forked and in-process execution mode.

0002-Integration-test-for-preprocessor.-2-modules-one-pro.patch
This patch adds an integration test for preprocessor feature. The integration test uses a 2 module setup, where the first module implements the TestListPreprocessor interface which selects only even indexed test classes from the list passed in. The second module(which depends on first) uses the artifact from first and uses the following configuration:
--------------------
<configuration>
<testPreprocessor>test.preprocessor.EvenTestOnlyPreprocessor[org.apache.maven.plugins.surefire:preprocessor-impl:1.0-SNAPSHOT]</testPreprocessor>
</configuration>
--------------------
The test then asserts that only 2(0th and 2nd) of the 4 test classes the second module has are executed.

The patches are created on a git-svn clone using format-patch, please use 'patch -p1 < 0001-Adds...'(while in the trunk directory) to apply the patches on svn repository. If using git-svn, git-am can be directly be invoked.

The patches are done over http://svn.apache.org/repos/asf/maven/surefire/trunk@1091357 (trunk HEAD) so should apply cleanly on any recent enough revision.

===============================================================================

Context:
http://stackoverflow.com/questions/5124823/reducing-the-build-time-hudson
http://code.google.com/p/tlb/issues/detail?id=1
http://test-load-balancer.github.com

Issue Links

• is depended upon by: SUREFIRE-528 Splitting tests names in two categories : success and failure
• is related to: SUREFIRE-1836 re-run should specify alternatives to the forking model
• is related to: SUREFIRE-701 showSuccess option for test reports: show reports only for failed tests
• is related to: SUREFIRE-713 Allow setting of excludes and includes via text files
• is related to: SUREFIRE-131 Excluding tests with command line pattern
• is related to: SUREFIRE-526 Better support for other plugins to determine which tests are included/excluded and the order the tests get run
• is related to: SUREFIRE-528 Splitting tests names in two categories : success and failure
• is related to: SUREFIRE-530 Allow runtime ordering of tests to be specified
• is related to: SUREFIRE-561 after running test, when tests fail, it's hard to the find the failure reason
• is related to: SUREFIRE-632 add a test-failed goal, to only run the tests which failed last time
• is related to: SUREFIRE-582 1. Define "set of tests" that should be run at the begining. 2. Fail build if any error/failure occured in some specified "set of tests".