[MNG-5872] Maven results are wrong after rerun test cases by RetryListeners

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Incomplete
• Component/s: Errors
• Assignee: Unassigned
• Reporter: Vishnu
• Created: 13/Aug/15 03:25
• Updated: 16/Apr/20 21:35
• Resolved: 16/Apr/20 21:35

Description

I am getting wrong result count after re-run the failed test case using RetryListener.

Environment details:

Java,
Selenium with TestNG
Chrome browser
Maven mode execution
Total Running Test case : 1

Simulation:

1. Have added Listener class in TestNG and running test case using maven

<listeners>
<listener class-name="<Package>.RetryListener" />
</listeners>

2. Made test case failed and it invoked failed test case and 2nd time, it got passed. The results would be as follows;

Expected Results ==> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 (Correct)

Actual Results ==> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 (Wrong)

I've used below classes for Listeners and calling RetryListener class from TestNG.xml and executing them from Maven mode.

IRetryAnalyzer class:

public class RetryAnalyzer implements IRetryAnalyzer {

private int retryCount = 0;
private int retryMaxCount = 1;

public RetryAnalyzer()

{ setCount(retryMaxCount); }

@Override
public boolean retry(ITestResult result) {
if (!result.isSuccess()) {
if (retryCount < retryMaxCount)

{ retryCount++; result.setStatus(ITestResult.SUCCESS); String message = Thread.currentThread().getName() + ": Error in " + result.getName() + " Retrying " + (retryMaxCount + 1 - retryCount) + " more time(s)"; System.out.println(message); Reporter.log(message); return true; }

else

{ result.setStatus(ITestResult.FAILURE); }

}
return false;
}

public void setCount(int count)

{ retryMaxCount = count; }

RetryLister Class:

public class RetryListener implements IAnnotationTransformer {

@SuppressWarnings("rawtypes")
@Override
public void transform(ITestAnnotation arg0, Class arg1, Constructor arg2,
Method arg3) {
IRetryAnalyzer retry = arg0.getRetryAnalyzer();
if (retry == null)

{ arg0.setRetryAnalyzer(RetryAnalyzer.class); }

}

public void onFinish(ITestContext context) {
Iterator<ITestResult> failedTestCases =context.getFailedTests().getAllResults().iterator();
while (failedTestCases.hasNext()) {
System.out.println("failedTestCases");
ITestResult failedTestCase = failedTestCases.next();
ITestNGMethod method = failedTestCase.getMethod();
if (context.getFailedTests().getResults(method).size() > 1)

{ System.out.println("failed test case remove as dup:" + failedTestCase.getTestClass().toString()); failedTestCases.remove(); }

else {

if (context.getPassedTests().getResults(method).size() > 0)

{ System.out.println("failed test case remove as pass retry:" + failedTestCase.getTestClass().toString()); failedTestCases.remove(); }

}
}
}