Uploaded image for project: 'Maven Surefire'
  1. Maven Surefire
  2. SUREFIRE-1239

ExecutionException java.lang.RuntimeException: org.apache.maven.surefire.report.ReporterException: When writing xml report stdout/stderr (No such file or directory)

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.19.1
    • Fix Version/s: 2.20
    • Component/s: Maven Surefire Plugin
    • Labels:
      None

      Description

      Getting the following exception every few builds:

      [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project publisher-console: ExecutionException java.lang.RuntimeException: org.apache.maven.surefire.report.ReporterException: When writing xml report stdout/stderr: /tmp/stdout9186853534312229181deferred (No such file or directory) -> [Help 1]

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jsdima closed the pull request at:

          https://github.com/apache/maven-surefire/pull/137

          Show
          githubbot ASF GitHub Bot added a comment - Github user jsdima closed the pull request at: https://github.com/apache/maven-surefire/pull/137
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jsdima commented on the issue:

          https://github.com/apache/maven-surefire/pull/137

          you are welcome

          Show
          githubbot ASF GitHub Bot added a comment - Github user jsdima commented on the issue: https://github.com/apache/maven-surefire/pull/137 you are welcome
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build maven-surefire #1660 (See https://builds.apache.org/job/maven-surefire/1660/)
          SUREFIRE-1239 ExecutionException java.lang.RuntimeException: (tibor17: http://git-wip-us.apache.org/repos/asf/?p=maven-surefire.git&a=commit&h=9dd4074e83c0edae5e2050f66e9cadfdba40fe01)

          • (edit) maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/TestSetRunListener.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build maven-surefire #1660 (See https://builds.apache.org/job/maven-surefire/1660/ ) SUREFIRE-1239 ExecutionException java.lang.RuntimeException: (tibor17: http://git-wip-us.apache.org/repos/asf/?p=maven-surefire.git&a=commit&h=9dd4074e83c0edae5e2050f66e9cadfdba40fe01 ) (edit) maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/report/TestSetRunListener.java
          Show
          tibor17 Tibor Digana added a comment - https://git-wip-us.apache.org/repos/asf?p=maven-surefire.git;a=commitdiff;h=9dd4074e83c0edae5e2050f66e9cadfdba40fe01
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Tibor17 commented on the issue:

          https://github.com/apache/maven-surefire/pull/137

          @jsdima
          Thx for contributing. Pls close this PR.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Tibor17 commented on the issue: https://github.com/apache/maven-surefire/pull/137 @jsdima Thx for contributing. Pls close this PR.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Tibor17 commented on the issue:

          https://github.com/apache/maven-surefire/pull/137

          @jsdima
          Long time I was thinking about deleting the class `ThreadedStreamConsumer`. It looks like old mechanism where data was populated from multiple `ForkStarter`s but this is no longer valid. Each method `ForkStarter#fork()` L532 already uses single Thread started by `executeCommandLineAsCallable()`. It means that `ThreadedStreamConsumer` the plugin does not benefit from the pumper and the blocking queue. WDYT?

          Show
          githubbot ASF GitHub Bot added a comment - Github user Tibor17 commented on the issue: https://github.com/apache/maven-surefire/pull/137 @jsdima Long time I was thinking about deleting the class `ThreadedStreamConsumer`. It looks like old mechanism where data was populated from multiple `ForkStarter`s but this is no longer valid. Each method `ForkStarter#fork()` L532 already uses single Thread started by `executeCommandLineAsCallable()`. It means that `ThreadedStreamConsumer` the plugin does not benefit from the pumper and the blocking queue. WDYT?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Tibor17 commented on the issue:

          https://github.com/apache/maven-surefire/pull/137

          @jsdima
          The constructor of `BlockingQueue` takes capacity of 2xE09 which practically means no limitation.
          The only operation which can block is `queue.take()`. What call sequences may cause that the `ForkClient` hangs?

          Show
          githubbot ASF GitHub Bot added a comment - Github user Tibor17 commented on the issue: https://github.com/apache/maven-surefire/pull/137 @jsdima The constructor of `BlockingQueue` takes capacity of 2xE09 which practically means no limitation. The only operation which can block is `queue.take()`. What call sequences may cause that the `ForkClient` hangs?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jsdima commented on the issue:

          https://github.com/apache/maven-surefire/pull/137

          @Tibor17

          >I am interested in this problem because as you said the plugin hangs. Why the plugin hanged, do you have explanation?

          It depends on a size of `private final BlockingQueue<String> items`
          in `ThreadedStreamConsumer` at the moment when Exception happens and amount of tests output .

          Surefire plugin has 3 important threads when running tests in separate process:

          • `StreamPumper` thread which reads input from child process and put them to `items`, it finishes when there is no more input from child process
          • `ThreadedStreamConsumer.Pumper`thread which takes line from `items` and call `ForkClient`to process it
          • `Main` thread which awaits when `StreamPumper` finishes

          so when exception happens `ThreadedStreamConsumer.Pumper`thread ends, but `StreamPumper` still reads input and put in into `items`, but when size of `items` becomes bigger than `ITEM_LIMIT_BEFORE_SLEEP`, this thread starts sleep every iteration:

          ```java
          if ( items.size() > ITEM_LIMIT_BEFORE_SLEEP )
          {
          try

          { Thread.sleep( 100 ); }

          catch ( InterruptedException ignore )
          {
          }
          }
          ```

          > Did you run the build with your patch? It means mvn -P run-its install.

          Yes. The result:
          ```
          [INFO] ------------------------------------------------------------------------
          [INFO] Reactor Summary:
          [INFO]
          [INFO] Apache Maven Surefire .............................. SUCCESS [ 10.423 s]
          [INFO] SureFire Logger API ................................ SUCCESS [ 3.082 s]
          [INFO] ShadeFire JUnit3 Provider .......................... SUCCESS [ 1.926 s]
          [INFO] SureFire API ....................................... SUCCESS [ 4.787 s]
          [INFO] SureFire Booter .................................... SUCCESS [ 2.016 s]
          [INFO] Maven Surefire Test-Grouping Support ............... SUCCESS [ 1.859 s]
          [INFO] SureFire Providers ................................. SUCCESS [ 0.650 s]
          [INFO] Shared JUnit3 Provider Code ........................ SUCCESS [ 0.870 s]
          [INFO] Shared Java 5 Provider Base ........................ SUCCESS [ 1.300 s]
          [INFO] Shared JUnit4 Provider Code ........................ SUCCESS [ 1.164 s]
          [INFO] Shared JUnit48 Provider Code ....................... SUCCESS [ 2.048 s]
          [INFO] SureFire JUnit Runner .............................. SUCCESS [ 0.857 s]
          [INFO] SureFire JUnit4 Runner ............................. SUCCESS [ 1.505 s]
          [INFO] Maven Surefire Common .............................. SUCCESS [ 7.801 s]
          [INFO] SureFire JUnitCore Runner .......................... SUCCESS [ 51.535 s]
          [INFO] SureFire TestNG Utils .............................. SUCCESS [ 1.158 s]
          [INFO] SureFire TestNG Runner ............................. SUCCESS [ 1.222 s]
          [INFO] Surefire Report Parser ............................. SUCCESS [ 1.460 s]
          [INFO] Maven Surefire Plugin .............................. SUCCESS [ 5.049 s]
          [INFO] Maven Failsafe Plugin .............................. SUCCESS [ 47.971 s]
          [INFO] Maven Surefire Report Plugin ....................... SUCCESS [ 8.128 s]
          [INFO] Maven Surefire Integration Test Setup .............. SUCCESS [ 9.125 s]
          [INFO] Maven Surefire Integration Tests ................... SUCCESS [47:52 min]
          [INFO] ------------------------------------------------------------------------
          [INFO] BUILD SUCCESS
          [INFO] ------------------------------------------------------------------------
          [INFO] Total time: 50:39 min
          [INFO] Finished at: 2016-12-30T14:34:19+03:00
          [INFO] Final Memory: 79M/1118M
          [INFO] ------------------------------------------------------------------------
          ```

          > The issue [1] you pointed out happens after your fix or without it?

          before applying the fix

          Show
          githubbot ASF GitHub Bot added a comment - Github user jsdima commented on the issue: https://github.com/apache/maven-surefire/pull/137 @Tibor17 >I am interested in this problem because as you said the plugin hangs. Why the plugin hanged, do you have explanation? It depends on a size of `private final BlockingQueue<String> items` in `ThreadedStreamConsumer` at the moment when Exception happens and amount of tests output . Surefire plugin has 3 important threads when running tests in separate process: `StreamPumper` thread which reads input from child process and put them to `items`, it finishes when there is no more input from child process `ThreadedStreamConsumer.Pumper`thread which takes line from `items` and call `ForkClient`to process it `Main` thread which awaits when `StreamPumper` finishes so when exception happens `ThreadedStreamConsumer.Pumper`thread ends, but `StreamPumper` still reads input and put in into `items`, but when size of `items` becomes bigger than `ITEM_LIMIT_BEFORE_SLEEP`, this thread starts sleep every iteration: ```java if ( items.size() > ITEM_LIMIT_BEFORE_SLEEP ) { try { Thread.sleep( 100 ); } catch ( InterruptedException ignore ) { } } ``` > Did you run the build with your patch? It means mvn -P run-its install. Yes. The result: ``` [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Maven Surefire .............................. SUCCESS [ 10.423 s] [INFO] SureFire Logger API ................................ SUCCESS [ 3.082 s] [INFO] ShadeFire JUnit3 Provider .......................... SUCCESS [ 1.926 s] [INFO] SureFire API ....................................... SUCCESS [ 4.787 s] [INFO] SureFire Booter .................................... SUCCESS [ 2.016 s] [INFO] Maven Surefire Test-Grouping Support ............... SUCCESS [ 1.859 s] [INFO] SureFire Providers ................................. SUCCESS [ 0.650 s] [INFO] Shared JUnit3 Provider Code ........................ SUCCESS [ 0.870 s] [INFO] Shared Java 5 Provider Base ........................ SUCCESS [ 1.300 s] [INFO] Shared JUnit4 Provider Code ........................ SUCCESS [ 1.164 s] [INFO] Shared JUnit48 Provider Code ....................... SUCCESS [ 2.048 s] [INFO] SureFire JUnit Runner .............................. SUCCESS [ 0.857 s] [INFO] SureFire JUnit4 Runner ............................. SUCCESS [ 1.505 s] [INFO] Maven Surefire Common .............................. SUCCESS [ 7.801 s] [INFO] SureFire JUnitCore Runner .......................... SUCCESS [ 51.535 s] [INFO] SureFire TestNG Utils .............................. SUCCESS [ 1.158 s] [INFO] SureFire TestNG Runner ............................. SUCCESS [ 1.222 s] [INFO] Surefire Report Parser ............................. SUCCESS [ 1.460 s] [INFO] Maven Surefire Plugin .............................. SUCCESS [ 5.049 s] [INFO] Maven Failsafe Plugin .............................. SUCCESS [ 47.971 s] [INFO] Maven Surefire Report Plugin ....................... SUCCESS [ 8.128 s] [INFO] Maven Surefire Integration Test Setup .............. SUCCESS [ 9.125 s] [INFO] Maven Surefire Integration Tests ................... SUCCESS [47:52 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 50:39 min [INFO] Finished at: 2016-12-30T14:34:19+03:00 [INFO] Final Memory: 79M/1118M [INFO] ------------------------------------------------------------------------ ``` > The issue [1] you pointed out happens after your fix or without it? before applying the fix
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Tibor17 commented on the issue:

          https://github.com/apache/maven-surefire/pull/137

          @jsdima
          I am interested in this problem because as you said the plugin hangs. Why the plugin hanged, do you have explanation?
          Did you run the build with your patch? It means `mvn -P run-its install`.
          The issue [1] you pointed out happens after your fix or without it?
          [1] `[ERROR] Failed to execute goal...`

          Show
          githubbot ASF GitHub Bot added a comment - Github user Tibor17 commented on the issue: https://github.com/apache/maven-surefire/pull/137 @jsdima I am interested in this problem because as you said the plugin hangs. Why the plugin hanged, do you have explanation? Did you run the build with your patch? It means `mvn -P run-its install`. The issue [1] you pointed out happens after your fix or without it? [1] ` [ERROR] Failed to execute goal...`
          Hide
          tibor17 Tibor Digana added a comment -

          Yoram Michaeli
          What was wrong with the tmp file?
          Why you closed this and what should I do with the PR?

          Show
          tibor17 Tibor Digana added a comment - Yoram Michaeli What was wrong with the tmp file? Why you closed this and what should I do with the PR?
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user jsdima opened a pull request:

          https://github.com/apache/maven-surefire/pull/137

          Fix SUREFIRE-1239

          Project to reproduce - https://github.com/jsdima/SUREFIRE-1239-reproduce

          More info:
          Problem appears when there are a lot of console output after last test(which leads to save this output to file) in TestSet, and in the next TestSet some test fails. It happens because stream (`Utf8RecodingDeferredFileOutputStream testStdOut`) which collects data after test execution but before TestSet finished then also used in next TestSet.

          Sometimes it leads to
          ```
          [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project SUREFIRE-1239-reproduce: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: java.lang.RuntimeException: org.apache.maven.surefire.report.ReporterException: When writing xml report stdout/stderr: /tmp/stdout8948920314382993444deferred (No such file or directory) -> [Help 1]
          ```

          sometimes build just hangs

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/jsdima/maven-surefire master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/maven-surefire/pull/137.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #137


          commit 768bd3c1bc0894269b6f6819eb643ea328d08c42
          Author: dg <dg@utgard>
          Date: 2016-12-23T11:14:17Z

          Fix SUREFIRE-1239


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user jsdima opened a pull request: https://github.com/apache/maven-surefire/pull/137 Fix SUREFIRE-1239 Project to reproduce - https://github.com/jsdima/SUREFIRE-1239-reproduce More info: Problem appears when there are a lot of console output after last test(which leads to save this output to file) in TestSet, and in the next TestSet some test fails. It happens because stream (`Utf8RecodingDeferredFileOutputStream testStdOut`) which collects data after test execution but before TestSet finished then also used in next TestSet. Sometimes it leads to ``` [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test (default-test) on project SUREFIRE-1239 -reproduce: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.19.1:test failed: java.lang.RuntimeException: org.apache.maven.surefire.report.ReporterException: When writing xml report stdout/stderr: /tmp/stdout8948920314382993444deferred (No such file or directory) -> [Help 1] ``` sometimes build just hangs You can merge this pull request into a Git repository by running: $ git pull https://github.com/jsdima/maven-surefire master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/maven-surefire/pull/137.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #137 commit 768bd3c1bc0894269b6f6819eb643ea328d08c42 Author: dg <dg@utgard> Date: 2016-12-23T11:14:17Z Fix SUREFIRE-1239
          Hide
          jsdima Dima Genov added a comment -

          Tibor Digana please reopen this issue.
          Project to reproduce - https://github.com/jsdima/SUREFIRE-1239-reproduce

          Show
          jsdima Dima Genov added a comment - Tibor Digana please reopen this issue. Project to reproduce - https://github.com/jsdima/SUREFIRE-1239-reproduce
          Hide
          tibor17 Tibor Digana added a comment -

          missing attached project and logs

          Show
          tibor17 Tibor Digana added a comment - missing attached project and logs
          Hide
          tibor17 Tibor Digana added a comment -

          Yoram Michaeli
          Pls attach the log file from command
          mvn -e -X test
          and the project you run, thx.

          Show
          tibor17 Tibor Digana added a comment - Yoram Michaeli Pls attach the log file from command mvn -e -X test and the project you run, thx.

            People

            • Assignee:
              tibor17 Tibor Digana
              Reporter:
              yorammi Yoram Michaeli
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development