Maven Surefire
  1. Maven Surefire
  2. SUREFIRE-827

Surefire 2.12 cannot run a single test, regression from 2.11

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.12
    • Fix Version/s: 2.12.1
    • Labels:
      None
    • Environment:
      Ubuntu 11.10

      Description

      1. Surefire 2.11
        $ mvn test -Dtest=DataTest#testDataServerGetNonExistentFile
        ...
        Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
      1. Surefire 2.12
        mvn test -Dtest=DataTest#testDataServerGetNonExistentFile
        ...
        Tests run: 9, Failures: 0, Errors: 0, Skipped: 0
      1. BUG-827.zip
        2 kB
        Falko Modler

        Issue Links

          Activity

          Hide
          Andrew Gaul added a comment -

          Verified fixed in r1364394. Thank you for your persistence Kristian.

          Show
          Andrew Gaul added a comment - Verified fixed in r1364394. Thank you for your persistence Kristian.
          Hide
          Kristian Rosenvold added a comment -

          Fixed in r1364159

          Show
          Kristian Rosenvold added a comment - Fixed in r1364159
          Hide
          Kristian Rosenvold added a comment -

          This issue was caused by the ClassDemarcatingRunne introduced in SUREFIRE-800, which cased a whole host of issues. Removed CDR

          Show
          Kristian Rosenvold added a comment - This issue was caused by the ClassDemarcatingRunne introduced in SUREFIRE-800 , which cased a whole host of issues. Removed CDR
          Hide
          Andrew Gaul added a comment -

          Kristian, going back to my example on 06 Feb 2012 with Apache commons-lang 3.1:

          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.11
          ...
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.12
          ...
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0

          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.13-SNAPSHOT
          ...
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0

          Tested with maven-surefire-plugin-2.13-20120706.132117-19.jar .

          Show
          Andrew Gaul added a comment - Kristian, going back to my example on 06 Feb 2012 with Apache commons-lang 3.1: $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.11 ... Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.12 ... Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.13-SNAPSHOT ... Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 Tested with maven-surefire-plugin-2.13-20120706.132117-19.jar .
          Hide
          Kristian Rosenvold added a comment -

          @Andrew; I am unable to reproduce this on my linux mint installation. Unless you can supply a mini test-project (maybe modify the one already attached) that demonstrates this I will close this issue as cannot reproduce.

          All the ms-windows users chiming in on this issue should note this has been fixed for windows.

          FWIW I tried this pom:

          <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
          <modelVersion>4.0.0</modelVersion>
          <groupId>SUREFIRE</groupId>
          <artifactId>BUG-827</artifactId>
          <version>0.0.1-SNAPSHOT</version>
          <dependencies>
          <dependency>
          <groupId>junit</groupId>
          <artifactId>junit</artifactId>
          <version>4.10</version>
          </dependency>
          </dependencies>
          <properties><surefire.version>2.12</surefire.version></properties>
          <build>
          <pluginManagement>
          <plugins>
          <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>$

          {surefire.version}

          </version>
          <configuration>
          <parallel>classes</parallel>
          <threadCount>2</threadCount>
          </configuration>
          </plugin>
          </plugins>
          </pluginManagement>
          </build>
          </project>

          Show
          Kristian Rosenvold added a comment - @Andrew; I am unable to reproduce this on my linux mint installation. Unless you can supply a mini test-project (maybe modify the one already attached) that demonstrates this I will close this issue as cannot reproduce. All the ms-windows users chiming in on this issue should note this has been fixed for windows. FWIW I tried this pom: <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd "> <modelVersion>4.0.0</modelVersion> <groupId>SUREFIRE</groupId> <artifactId>BUG-827</artifactId> <version>0.0.1-SNAPSHOT</version> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.10</version> </dependency> </dependencies> <properties><surefire.version>2.12</surefire.version></properties> <build> <pluginManagement> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>$ {surefire.version} </version> <configuration> <parallel>classes</parallel> <threadCount>2</threadCount> </configuration> </plugin> </plugins> </pluginManagement> </build> </project>
          Hide
          Falko Modler added a comment -

          Works on Windows 7 x64.

          Show
          Falko Modler added a comment - Works on Windows 7 x64.
          Hide
          Andrew Gaul added a comment -

          This still fails for me with 2.13-20120620.212637-15 running on Ubuntu 12.04.

          Show
          Andrew Gaul added a comment - This still fails for me with 2.13-20120620.212637-15 running on Ubuntu 12.04.
          Hide
          Kristian Rosenvold added a comment - - edited

          ANd yes, it's fixed on trunk ATM even though 824 is still open.

          Show
          Kristian Rosenvold added a comment - - edited ANd yes, it's fixed on trunk ATM even though 824 is still open.
          Hide
          Kristian Rosenvold added a comment -

          This issue was fixed by the fix for SUREFIRE-824, which re-added support for redmond-based OS'es.

          Show
          Kristian Rosenvold added a comment - This issue was fixed by the fix for SUREFIRE-824 , which re-added support for redmond-based OS'es.
          Hide
          Gabriele Catania added a comment -

          It works on my linux installation, but doesn't on my colleagues' windows ones.

          we are using TestNG with surefire 2.12.

          Show
          Gabriele Catania added a comment - It works on my linux installation, but doesn't on my colleagues' windows ones. we are using TestNG with surefire 2.12.
          Hide
          Falko Modler added a comment -

          Any progress? In my opinion this is a showstopper (for 2.13).

          Show
          Falko Modler added a comment - Any progress? In my opinion this is a showstopper (for 2.13).
          Hide
          Sebb added a comment -

          I can confirm that the problem is not related to the JUnit version.
          SureFire 2.11 works fine with JUnit 4.7, 4.8, 4.9, 4.10 (and even 3.8.2)
          SureFire 2.12 does not work with any of them.
          For me, the -Dparallel setting makes no difference to 2.12.

          Show
          Sebb added a comment - I can confirm that the problem is not related to the JUnit version. SureFire 2.11 works fine with JUnit 4.7, 4.8, 4.9, 4.10 (and even 3.8.2) SureFire 2.12 does not work with any of them. For me, the -Dparallel setting makes no difference to 2.12.
          Hide
          Andrew Gaul added a comment -

          2.13-SNAPSHOT (2.13-20120228.152625-12) still broken with JUnit.

          Show
          Andrew Gaul added a comment - 2.13-SNAPSHOT (2.13-20120228.152625-12) still broken with JUnit.
          Hide
          Andreas Kuhtz added a comment -

          Works with current 2.13-SNAPSHOT (maven-surefire-plugin-2.13-20120228.152623-12 ....) with testNG.

          Show
          Andreas Kuhtz added a comment - Works with current 2.13-SNAPSHOT (maven-surefire-plugin-2.13-20120228.152623-12 ....) with testNG.
          Hide
          Kristian Rosenvold added a comment -

          The problem is reproduced and "in progress", although not entirely trivial to solve.

          AFAIK this only applies when using the 4.7 JUnit provider, so unless someone has additional info, there is no need to report any more testcases.

          Show
          Kristian Rosenvold added a comment - The problem is reproduced and "in progress", although not entirely trivial to solve. AFAIK this only applies when using the 4.7 JUnit provider, so unless someone has additional info, there is no need to report any more testcases.
          Hide
          Falko Modler added a comment -

          This very basic maven project can be used to reproduce the issue.

          It has one simple Test (surefire.bug_827.SomeTest) which runs just fine when executing mvn test.

          But when mvn test -Dtest=SomeTest is executed, surefire raises an error: No tests were executed!

          Junit 4.10 is defined in the pom.xml.
          I used Maven 3.0.4 and JDK6u29 on Win7 Home Premium x64.

          When going back to surefire 2.11, everything is fine!

          Show
          Falko Modler added a comment - This very basic maven project can be used to reproduce the issue. It has one simple Test (surefire.bug_827.SomeTest) which runs just fine when executing mvn test . But when mvn test -Dtest=SomeTest is executed, surefire raises an error: No tests were executed! Junit 4.10 is defined in the pom.xml. I used Maven 3.0.4 and JDK6u29 on Win7 Home Premium x64. When going back to surefire 2.11, everything is fine!
          Hide
          François LEIBER added a comment -

          You're right, I read the ticket title a bit quickly, I'm creating another one (and yes, it happens all the time).

          Show
          François LEIBER added a comment - You're right, I read the ticket title a bit quickly, I'm creating another one (and yes, it happens all the time).
          Hide
          Kristian Rosenvold added a comment -

          Francois; This is a separate issue from what the others are reporting.

          The lines of code you have found are not the problem, but I am very interested in the actual issue. The code that seems to be failing is new for 2.12, so there would seem to be some kind of unforseen race condition or similar. If you file a separate issue on this one I'd be very interested in your surefire configuration and maybe the log output from running with mvn -X . Does this failure happen all the time ?

          Please file a different ticket.

          Show
          Kristian Rosenvold added a comment - Francois; This is a separate issue from what the others are reporting. The lines of code you have found are not the problem, but I am very interested in the actual issue. The code that seems to be failing is new for 2.12, so there would seem to be some kind of unforseen race condition or similar. If you file a separate issue on this one I'd be very interested in your surefire configuration and maybe the log output from running with mvn -X . Does this failure happen all the time ? Please file a different ticket.
          Hide
          Deivydas added a comment - - edited

          We have the same issue (-Dtest=test1,test2 doesn't run any test) as Falko (maven 3.0.3/surefire 2.12/junit-4.10) doesn't matter parallel='classes' or parallel='none'. Switching to 2.11 works.

          Show
          Deivydas added a comment - - edited We have the same issue (-Dtest=test1,test2 doesn't run any test) as Falko (maven 3.0.3/surefire 2.12/junit-4.10) doesn't matter parallel='classes' or parallel='none'. Switching to 2.11 works.
          Hide
          François LEIBER added a comment - - edited

          We also have a big issue with surefire 2.12 (using default configuration, which means forkMode=once): the Junit tests of the first module pass perfectly, then we have a "Z,0,BYE!" in the log and the surefire plugin fails with:

          The forked VM terminated without saying properly goodbye. VM crash or System.exit called ?
                  at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.close(ForkClient.java:244)
          

          If I check maven/surefire/trunk/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java, I see the following lines:

           	runSuitesInProcess( testSet, testClassLoader, startupConfiguration, providerConfiguration );
           	// Say bye.
           	System.out.println("Z,0,BYE!");
           	System.out.flush();
           	// noinspection CallToSystemExit
           	System.exit( 0 );
          

          Why is surefire surprised someone called System.exit() if he's the one who did it?

          Show
          François LEIBER added a comment - - edited We also have a big issue with surefire 2.12 (using default configuration, which means forkMode=once): the Junit tests of the first module pass perfectly, then we have a "Z,0,BYE!" in the log and the surefire plugin fails with: The forked VM terminated without saying properly goodbye. VM crash or System .exit called ? at org.apache.maven.plugin.surefire.booterclient.output.ForkClient.close(ForkClient.java:244) If I check maven/surefire/trunk/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java, I see the following lines: runSuitesInProcess( testSet, testClassLoader, startupConfiguration, providerConfiguration ); // Say bye. System .out.println( "Z,0,BYE!" ); System .out.flush(); // noinspection CallToSystemExit System .exit( 0 ); Why is surefire surprised someone called System.exit() if he's the one who did it?
          Hide
          Andrew Gaul added a comment - - edited

          Oliver I believe that this is related to parallel tests and is a regression from SUREFIRE-760. Given an Apache commons-lang 3.1 project and setting the surefire version in the pom to $

          {surefire.version}

          , I get these results:

          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.11
          ...
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
          
          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.12
          ...
          Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
          
          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=none -Dsurefire.version=2.11
          ...
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
          
          $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=none -Dsurefire.version=2.12
          ...
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
          

          I cannot speak to Falko's symptoms.

          Show
          Andrew Gaul added a comment - - edited Oliver I believe that this is related to parallel tests and is a regression from SUREFIRE-760 . Given an Apache commons-lang 3.1 project and setting the surefire version in the pom to $ {surefire.version} , I get these results: $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.11 ... Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=classes -Dsurefire.version=2.12 ... Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=none -Dsurefire.version=2.11 ... Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 $ mvn test -Dtest=ImmutablePairTest#testBasic -Dparallel=none -Dsurefire.version=2.12 ... Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 I cannot speak to Falko's symptoms.
          Hide
          Olivier Lamy (*$^¨%`£) added a comment -

          I have tested a project with junit 4.8.2 and this feature works.
          Do you have any sample project to reproduce ?

          Show
          Olivier Lamy (*$^¨%`£) added a comment - I have tested a project with junit 4.8.2 and this feature works. Do you have any sample project to reproduce ?
          Hide
          Holger Hoffstätte added a comment -

          Confirmed with maven 3.0.4/surefire 2.12/junit-4.10. Switching to 2.11 immediately works.

          Show
          Holger Hoffstätte added a comment - Confirmed with maven 3.0.4/surefire 2.12/junit-4.10. Switching to 2.11 immediately works.
          Hide
          Falko Modler added a comment -

          Junit.

          Show
          Falko Modler added a comment - Junit.
          Hide
          Olivier Lamy (*$^¨%`£) added a comment -

          junit or testng ?

          Show
          Olivier Lamy (*$^¨%`£) added a comment - junit or testng ?
          Hide
          Falko Modler added a comment - - edited

          Similar problem here:

          -Dtest=SomeTest does not run any test an raises an error:

          ...
          Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
          
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD FAILURE
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 4:20.351s
          [INFO] Finished at: Thu Feb 02 23:58:33 CET 2012
          [INFO] Final Memory: 17M/981M
          [INFO] ------------------------------------------------------------------------
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test (default-test) on project someproject: No tests were executed!  (Set -DfailIfNoTests=false to ignore this error.) -> [Help 1]
          

          I tried to debug the problem bu got lost in ForkStarter.java which received a lot of changes since 2.11.

          Env: Java6 u29, Maven 3.0.4.

          Show
          Falko Modler added a comment - - edited Similar problem here: -Dtest=SomeTest does not run any test an raises an error: ... Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 4:20.351s [INFO] Finished at: Thu Feb 02 23:58:33 CET 2012 [INFO] Final Memory: 17M/981M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12:test ( default -test) on project someproject: No tests were executed! (Set -DfailIfNoTests= false to ignore this error.) -> [Help 1] I tried to debug the problem bu got lost in ForkStarter.java which received a lot of changes since 2.11. Env: Java6 u29, Maven 3.0.4.

            People

            • Assignee:
              Kristian Rosenvold
              Reporter:
              Andrew Gaul
            • Votes:
              26 Vote for this issue
              Watchers:
              23 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development