Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1305

surefire fails on parallel tests when newline character is in test description

    Details

      Description

      zohhak runner may create test description with newline character in it. it runs without any problem with surefire and non-parallel build and with parallel build when there is no newline char in the description. furthermore it runs fine with plain junit in parallel mode

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build maven-surefire #1652 (See https://builds.apache.org/job/maven-surefire/1652/)
        SUREFIRE-1305 surefire fails on parallel tests when newline character (tibor17: rev d7bf6ce49043a3301494fdd26c538afad4e1938d)

        • (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4RunListener.java
        • (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4StackTraceWriter.java
        • (add) surefire-api/src/main/java/org/apache/maven/surefire/util/internal/TestClassMethodNameUtils.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build maven-surefire #1652 (See https://builds.apache.org/job/maven-surefire/1652/ ) SUREFIRE-1305 surefire fails on parallel tests when newline character (tibor17: rev d7bf6ce49043a3301494fdd26c538afad4e1938d) (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4RunListener.java (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4StackTraceWriter.java (add) surefire-api/src/main/java/org/apache/maven/surefire/util/internal/TestClassMethodNameUtils.java
        Show
        tibor17 Tibor Digana added a comment - https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=commit;h=d7bf6ce49043a3301494fdd26c538afad4e1938d
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build maven-surefire #1649 (See https://builds.apache.org/job/maven-surefire/1649/)
        SUREFIRE-1305 surefire fails on parallel tests when newline character (tibor17: rev c2a7d4b7f28b0fb5456b9871fd87cbac5c70cbd3)

        • (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4RunListener.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build maven-surefire #1649 (See https://builds.apache.org/job/maven-surefire/1649/ ) SUREFIRE-1305 surefire fails on parallel tests when newline character (tibor17: rev c2a7d4b7f28b0fb5456b9871fd87cbac5c70cbd3) (edit) surefire-providers/common-junit4/src/main/java/org/apache/maven/surefire/common/junit4/JUnit4RunListener.java
        Show
        tibor17 Tibor Digana added a comment - https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=commit;h=c2a7d4b7f28b0fb5456b9871fd87cbac5c70cbd3
        Hide
        tibor17 Tibor Digana added a comment - - edited

        is there a simple way to run surefire programatically inside IDE? or how does surefire-team run tests for the surefire itself?

        Here is a hint in documentation:

        http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html

        It would be more simple for you to debug tests within Maven process and not in forked JVM. To do so add <forkCount>0</forkCount> in Surefire plugin configuration and debug afterwards.

        Show
        tibor17 Tibor Digana added a comment - - edited is there a simple way to run surefire programatically inside IDE? or how does surefire-team run tests for the surefire itself? Here is a hint in documentation: http://maven.apache.org/surefire/maven-surefire-plugin/examples/debugging.html It would be more simple for you to debug tests within Maven process and not in forked JVM. To do so add <forkCount>0</forkCount> in Surefire plugin configuration and debug afterwards.
        Hide
        piotr.turski Piotr Turski added a comment -

        any links to documentation on what's the interface of `Description`? what exactly should be satisfied? if it runs correctly in non-parallel model, shouldn't it also work in parallel mode?

        Is the runner mutating the description?

        mutating? runner creates it. it creates parameterized tests.

        is there a simple way to run surefire programatically inside IDE? or how does surefire-team run tests for the surefire itself?

        Show
        piotr.turski Piotr Turski added a comment - any links to documentation on what's the interface of `Description`? what exactly should be satisfied? if it runs correctly in non-parallel model, shouldn't it also work in parallel mode? Is the runner mutating the description? mutating? runner creates it. it creates parameterized tests. is there a simple way to run surefire programatically inside IDE? or how does surefire-team run tests for the surefire itself?
        Hide
        tibor17 Tibor Digana added a comment -

        Piotr Turski
        I guess the custom runner does not satisfy the interface of Description. We have to extract method and class from it which is important to run method and/or class in parallel.
        Try to attach Surefire source, run it in debug mode and debug classes ConcurrentRunListener and JUnitCoreRunListener. There is no TestSet for Description. Is the runner mutating the description?

        You should debug the method fillTestCountMap in class JUnitCoreRunListener. Either the class name does not come to the map classMethodCounts or the description of class was mutated before ConcurrentRunListener#testStarting().
        The JUnit description must be parsed successfully in JUnit4RunListener#testStarted( Description description ).

        Show
        tibor17 Tibor Digana added a comment - Piotr Turski I guess the custom runner does not satisfy the interface of Description . We have to extract method and class from it which is important to run method and/or class in parallel. Try to attach Surefire source, run it in debug mode and debug classes ConcurrentRunListener and JUnitCoreRunListener . There is no TestSet for Description . Is the runner mutating the description? You should debug the method fillTestCountMap in class JUnitCoreRunListener . Either the class name does not come to the map classMethodCounts or the description of class was mutated before ConcurrentRunListener#testStarting() . The JUnit description must be parsed successfully in JUnit4RunListener#testStarted( Description description ) .
        Hide
        piotr.turski Piotr Turski added a comment - - edited

        still a bug but a bit different output. i took the project that reproduces the issue, added:

           <pluginRepositories>
                <pluginRepository>
                    <id>apache.snapshots</id>
                    <url>http://repository.apache.org/snapshots/</url>
                </pluginRepository>
            </pluginRepositories>
        

        and changed version to

         <version>2.19.2-SNAPSHOT</version>
        

        and after running tests:

        [INFO] Scanning for projects...
        [INFO]                                                                         
        [INFO] ------------------------------------------------------------------------
        [INFO] Building playground 1.0-SNAPSHOT
        [INFO] ------------------------------------------------------------------------
        [INFO] 
        [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ playground ---
        [debug] execute contextualize
        [INFO] Using 'UTF-8' encoding to copy filtered resources.
        [INFO] skip non existing resourceDirectory /xxxxxxxxx/zohhak_parallel_bug/src/main/resources
        [INFO] 
        [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ playground ---
        [INFO] No sources to compile
        [INFO] 
        [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ playground ---
        [debug] execute contextualize
        [INFO] Using 'UTF-8' encoding to copy filtered resources.
        [INFO] skip non existing resourceDirectory /xxxxxxxx/zohhak_parallel_bug/src/test/resources
        [INFO] 
        [INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @ playground ---
        [INFO] Nothing to compile - all classes are up to date
        [INFO] 
        [INFO] --- maven-surefire-plugin:2.19.2-SNAPSHOT:test (default-test) @ playground ---
        [INFO] 
        [INFO] -------------------------------------------------------
        [INFO]  T E S T S
        [INFO] -------------------------------------------------------
        [INFO] Running testMethod
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in testMethod
        [INFO] 
        [INFO] Results:
        [INFO] 
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
        [INFO] 
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 2.240s
        [INFO] Finished at: Sun Nov 20 23:01:58 CET 2016
        [INFO] Final Memory: 11M/303M
        [INFO] ------------------------------------------------------------------------
        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.2-SNAPSHOT:test (default-test) on project playground: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.2-SNAPSHOT:test failed: There was an error in the forked process
        [ERROR] org.apache.maven.surefire.testset.TestSetFailedException: Test mechanism :: null
        [ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.rethrowAnyTestMechanismFailures(JUnit4RunListener.java:210)
        [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:138)
        [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
        [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
        [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
        [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:157)
        [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:300)
        [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:251)
        [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:130)
        [ERROR] Caused by: java.lang.NullPointerException
        [ERROR] at org.apache.maven.surefire.junitcore.ConcurrentRunListener.testStarting(ConcurrentRunListener.java:134)
        [ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.testStarted(JUnit4RunListener.java:93)
        [ERROR] at org.junit.runner.notification.SynchronizedRunListener.testStarted(SynchronizedRunListener.java:49)
        [ERROR] at org.junit.runner.notification.RunNotifier$3.notifyListener(RunNotifier.java:121)
        [ERROR] at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
        [ERROR] at org.junit.runner.notification.RunNotifier.fireTestStarted(RunNotifier.java:118)
        [ERROR] at org.apache.maven.surefire.common.junit4.Notifier.fireTestStarted(Notifier.java:100)
        [ERROR] at org.junit.internal.runners.model.EachTestNotifier.fireTestStarted(EachTestNotifier.java:42)
        [ERROR] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:323)
        [ERROR] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        [ERROR] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        [ERROR] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        [ERROR] at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:393)
        [ERROR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        [ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        [ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        [ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        [ERROR] at java.lang.Thread.run(Thread.java:745)
        [ERROR] -> [Help 1]
        [ERROR] 
        [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
        [ERROR] Re-run Maven using the -X switch to enable full debug logging.
        [ERROR] 
        [ERROR] For more information about the errors and possible solutions, please read the following articles:
        [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
        
        Process finished with exit code 1
        
        
        Show
        piotr.turski Piotr Turski added a comment - - edited still a bug but a bit different output. i took the project that reproduces the issue, added: <pluginRepositories> <pluginRepository> <id> apache.snapshots </id> <url> http://repository.apache.org/snapshots/ </url> </pluginRepository> </pluginRepositories> and changed version to <version>2.19.2-SNAPSHOT</version> and after running tests: [INFO] Scanning for projects... [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building playground 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-resources-plugin:2.5:resources (default-resources) @ playground --- [debug] execute contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /xxxxxxxxx/zohhak_parallel_bug/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.5.1:compile (default-compile) @ playground --- [INFO] No sources to compile [INFO] [INFO] --- maven-resources-plugin:2.5:testResources (default-testResources) @ playground --- [debug] execute contextualize [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /xxxxxxxx/zohhak_parallel_bug/src/test/resources [INFO] [INFO] --- maven-compiler-plugin:3.5.1:testCompile (default-testCompile) @ playground --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- maven-surefire-plugin:2.19.2-SNAPSHOT:test (default-test) @ playground --- [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running testMethod [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in testMethod [INFO] [INFO] Results: [INFO] [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.240s [INFO] Finished at: Sun Nov 20 23:01:58 CET 2016 [INFO] Final Memory: 11M/303M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.2-SNAPSHOT:test (default-test) on project playground: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.2-SNAPSHOT:test failed: There was an error in the forked process [ERROR] org.apache.maven.surefire.testset.TestSetFailedException: Test mechanism :: null [ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.rethrowAnyTestMechanismFailures(JUnit4RunListener.java:210) [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:138) [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107) [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83) [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75) [ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:157) [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:300) [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:251) [ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:130) [ERROR] Caused by: java.lang.NullPointerException [ERROR] at org.apache.maven.surefire.junitcore.ConcurrentRunListener.testStarting(ConcurrentRunListener.java:134) [ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.testStarted(JUnit4RunListener.java:93) [ERROR] at org.junit.runner.notification.SynchronizedRunListener.testStarted(SynchronizedRunListener.java:49) [ERROR] at org.junit.runner.notification.RunNotifier$3.notifyListener(RunNotifier.java:121) [ERROR] at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72) [ERROR] at org.junit.runner.notification.RunNotifier.fireTestStarted(RunNotifier.java:118) [ERROR] at org.apache.maven.surefire.common.junit4.Notifier.fireTestStarted(Notifier.java:100) [ERROR] at org.junit.internal.runners.model.EachTestNotifier.fireTestStarted(EachTestNotifier.java:42) [ERROR] at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:323) [ERROR] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) [ERROR] at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) [ERROR] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [ERROR] at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:393) [ERROR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [ERROR] at java.lang.Thread.run(Thread.java:745) [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException Process finished with exit code 1
        Hide
        tibor17 Tibor Digana added a comment -

        Piotr Turski
        Can you check it out with Version 2.19.2-SNAPSHOT?
        Method #rethrowAnyTestMechanismFailures() changed and I think it should not throw NPE.

        Show
        tibor17 Tibor Digana added a comment - Piotr Turski Can you check it out with Version 2.19.2-SNAPSHOT? Method #rethrowAnyTestMechanismFailures() changed and I think it should not throw NPE.

          People

          • Assignee:
            tibor17 Tibor Digana
            Reporter:
            piotr.turski Piotr Turski
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development