[SUREFIRE-1058] TestSetFailedException trying to use maven-surefire-plugin + regular expressions for TestNG groups

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.16
• Fix Version/s: 2.17
• Component/s: TestNG support
• Assignee: Andreas Gudian
• Reporter: Tracey Ruark
• Created: 05/Feb/14 14:55
• Updated: 20/Feb/14 15:05
• Resolved: 10/Feb/14 14:54

Description

I have a Maven project with parent -> child poms. Each child project has TestNG tests. What I'd like to be able to do is use regular expressions on the TestNG group and execute tests across projects which match the given regex.

Running TestNG tests using a non-regular expression group across projects works just fine. For example this works:

mvn test -DargLine="-Denvironment=test" -Dgroups="foo.bar.something"

This runs the correct test methods which have that exact tag across all my child projects.

However if I do something like this:

mvn test -DargLine="-Denvironment=test" -Dgroups="foo.bar.*"

I get this exception:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test     (default-test) on project test-automation-ui: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test failed: There was an error in the forked process
[ERROR] org.apache.maven.surefire.testset.TestSetFailedException: null; nested exception is java.lang.reflect.InvocationTargetException: null
[ERROR] java.lang.reflect.InvocationTargetException
[ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[ERROR] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
[ERROR] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[ERROR] at java.lang.reflect.Method.invoke(Method.java:601)
[ERROR] at org.apache.maven.surefire.testng.TestNGExecutor.getGroupMatchingSelector(TestNGExecutor.java:172)
[ERROR] at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:66)
[ERROR] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeMulti(TestNGDirectoryTestSuite.java:204)
[ERROR] at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:107)
[ERROR] at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:113)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
[ERROR] Caused by: org.apache.maven.surefire.group.parse.TokenMgrError: Lexical error at line 1, column 13.  Encountered: <EOF> after : ""
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParserTokenManager.getNextToken(GroupMatcherParserTokenManager.java:468)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_scan_token(GroupMatcherParser.java:527)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_3_7(GroupMatcherParser.java:274)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_3R_3(GroupMatcherParser.java:287)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_3_3(GroupMatcherParser.java:279)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_3R_1(GroupMatcherParser.java:320)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_3_1(GroupMatcherParser.java:335)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.jj_2_1(GroupMatcherParser.java:179)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.expr(GroupMatcherParser.java:63)
[ERROR] at org.apache.maven.surefire.group.parse.GroupMatcherParser.parse(GroupMatcherParser.java:56)
[ERROR] at org.apache.maven.surefire.testng.utils.GroupMatcherMethodSelector.setGroups(GroupMatcherMethodSelector.java:83)
[ERROR] ... 12 more
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]

If I use the bypass surefire and run TestNG directly using same regular expression in a testng.xml suite file it works just fine:

<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >

<suite name="foosuite">
    <parameter name="environment"  value="dev"/>
    <test name="footest">
        <groups>
            <run>
                <include name="foo.bar.*"/>
            </run>
        </groups>
        <packages>
            <package name="com.foo.bar.somepackage.*"/>
        </packages>
    </test>
</suite>