[GROOVY-10878] Improve JaCoCo's branch code coverage of a Groovy assert statement

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.6
• Fix Version/s: 4.0.7
• Assignee: Eric Milles
• Reporter: Damir Murat
• Created: 17/Dec/22 09:45
• Updated: 10/Feb/23 21:39
• Resolved: 19/Dec/22 16:50

Description

At the moment, there is no way to have full branch coverage for Groovy assert statements. In a larger project, this is very distractive when trying to find the pieces of actual logic that should be better covered with tests. I believe a slight change in assert statement code generation can significantly improve the situation.

JaCoCo has long-standing issues with covering calls of methods that throw exceptions. When such methods are called inside of, if/else branches, for example, the result is partial coverage reported for those branches.

However, there is a JaCoCo idiom (https://github.com/jacoco/jacoco/issues/370#issuecomment-267854179) that can be used to avoid uncovered code in those cases. The basic idea is to create and return an exception from a called method and throw that exception from a caller, like in:

void fail() {
  throw create();
}

RuntimeException create() {
  return new RuntimeException();
}

How this relates to the Groovy assert statement? For example, for a simple assert statement like

assert firstName != null

Groovy generates something like

ValueRecorder var1 = new ValueRecorder();

try {
  String var10000 = this.firstName;
  var1.record(var10000, 8);
  var1.record(var10000, 8);
  if (var10000 != null) {
    var1.clear();
  } else {
    ScriptBytecodeAdapter.assertFailed(AssertionRenderer.render("assert firstName != null", var1), (Object)null);
  }

} catch (Throwable var3) {
  var1.clear();
  throw var3;
}

The problem with generated code is a

ScriptBytecodeAdapter.assertFailed(AssertionRenderer.render("assert firstName != null", var1), (Object)null);

method call. Inside that method, an exception is created and thrown. Since JaCoCo cannot cover that line completely, the branch

if (var10000 != null)

will be reported as partially covered.

To avoid those issues, ScriptBytecodeAdapter.assertFailed() can be adapted (or a new method can be introduced like in the example below) to return the exception instead of throwing it. And then, the calling generated code can throw that returned exception:

try {
  ...
  if (var10000 != null) {
    ...
  } else {
    Throwable throwable = ScriptBytecodeAdapter.createAssertionError(AssertionRenderer.render("assert firstName != null", var1), (Object)null);
    throw throwable
  }

} catch (Throwable var3) {
  ...
}

I have a small project demonstrating the issue and a possible solution here: https://github.com/dmurat/groovy-assert-jacoco-coverage-problem

Tnx