[SUREFIRE-1795] surefire-reports is not working in parallel

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-M4
• Component/s: Maven Surefire Plugin
• Environment: Centos 7 +JDK 1.8.0 + Maven 3.0.5 + surefire plugin 2.19.1 + testng 6.8; ; Centos 7 + JDK 1.8.0 + Maven 3.5.2 + surefire plugin 2.19.1 + testng 6.8; ; Windows 10 + JDK 1.8.0 + Maven 3.5.0 + surefirel plugin 3.0.0-M4 + testng 6.8
• Assignee: Unassigned
• Reporter: QIAO YU
• Created: 22/May/20 14:38
• Updated: 30/May/23 18:09

Description

Create a maven project, add 4 testng classes to print out logs, for example:

package com.verizon.rngverify;

import org.testng.annotations.Test;

public class testClass1 {  
    @Test public void test1() throws Exception { 
        System.out.println("class1method1"); 
        Thread.sleep(100); 
        System.out.println("class1method1 end"); 
    }
}

 

Create a testng xml file to control above 4 test cases in parallel with 4 threads:

<suite name="tsname" parallel="tests" thread-count="4" verbose="1">
  <test name="testname1_in_xml" > 
    <classes> 
      <class name="com.verizon.rngverify.testClass1"/> 
    </classes> 
  </test>  
  
  <test name="testname2_in_xml"> 
    <classes> 
      <class name="com.verizon.rngverify.testClass2"/> 
    </classes> 
  </test>  

  <test name="testname3_in_xml"> 
    <classes> 
      <class name="com.verizon.rngverify.testClass3" /> 
    </classes> 
  </test>  

  <test name="testname4_in_xml"> 
    <classes> 
      <class name="com.verizon.rngverify.testClass4" /> 
    </classes> 
  </test>
</suite>

 

Use surefire plugin in pom:

<plugins> 
  <plugin> 
    <groupId>org.apache.maven.plugins</groupId> 
    <artifactId>maven-compiler-plugin</artifactId> 
    <version>3.7.0</version> 
    <configuration> <source>1.8</source> <target>1.8</target> </configuration>      
  </plugin>
  <plugin> 
    <groupId>org.apache.maven.plugins</groupId> 
    <artifactId>maven-surefire-plugin</artifactId> 
    <version>3.0.0-M4</version> 
    <configuration> 
      <suiteXmlFiles> <file>src/test/resources/ts1234.xml</file> </suiteXmlFiles>   
    </configuration> 
   </plugin>
 </plugins>

Run above project:

mvn clean install

 

...

[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
class1method1
class2method1
class3method1
class4method1
class2method1 end
class3method1 end
class4method1 end
class1method1 end
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.617 s - in TestSuite

...

 

But in ${project}/target/surefire-reports/testng-results.xml, only 3 cases were found. Also under ${project}/target/surefire-reports/junitreports folder, only 3 files were generated.

 

In general, if the thread-count > 1, it happens randomly.