[SUREFIRE-1603] Pass TestReporter entries to the reports generated by Surefire

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.22.1, 3.0.0-M1
• Component/s: JUnit 5.x support
• Labels: junit5
• Assignee: Tibor Digana
• Reporter: John Knight
• Created: 22/Nov/18 20:11
• Updated: 27/Mar/22 23:23
• Resolved: 27/Mar/22 23:23

Description

Using JUnit 5.31 with the following configuration:

<configurationParameters>
    junit.jupiter.execution.parallel.enabled = true
    junit.jupiter.execution.parallel.config.dynamic.factor = 8
    junit.jupiter.extensions.autodetection.enabled = true
</configurationParameters>

and the following tests (for demonstration purposes)

public class AppTest
{
    @Test
    public void mogbyTrue()
    {
        for (int i = 0; i < 5; i++) {
            System.out.println("mogby " + Thread.currentThread().getName());
        }
        assertTrue( false );
    }
    @Test
    public void kermitTrue()
    {
        for (int i = 0; i < 5; i++) {
            System.out.println("kermit " + Thread.currentThread().getName());
        }
        assertTrue( false );
    }

When the tests are executed via mvn test, the messages from both tests appear under the output for the first test:

<testcase name="mogbyTrue" classname="com.foo.logtest.AppTest" time="0.013">
    <failure message="expected: ...
    </failure>
<system-out><![CDATA[mogby ForkJoinPool-1-worker-43
kermit ForkJoinPool-1-worker-50
mogby ForkJoinPool-1-worker-43
kermit ForkJoinPool-1-worker-50
mogby ForkJoinPool-1-worker-43
kermit ForkJoinPool-1-worker-50
mogby ForkJoinPool-1-worker-43
kermit ForkJoinPool-1-worker-50
mogby ForkJoinPool-1-worker-43
kermit ForkJoinPool-1-worker-50
]]></system-out>
</testcase>

If I change the code to use TestReporter.publishEntry(), the messages don't make their way into the surefire reports at all.

If I configure Capturing stdout - https://junit.org/junit5/docs/snapshot/user-guide/#running-tests-capturing-output - The output for both tests is still captured within the <system-out> field of the first test, but in a different order:

<system-out><![CDATA[kermit ForkJoinPool-1-worker-50
kermit ForkJoinPool-1-worker-50
kermit ForkJoinPool-1-worker-50
kermit ForkJoinPool-1-worker-50
kermit ForkJoinPool-1-worker-50
mogby ForkJoinPool-1-worker-43
mogby ForkJoinPool-1-worker-43
mogby ForkJoinPool-1-worker-43
mogby ForkJoinPool-1-worker-43
mogby ForkJoinPool-1-worker-43
]]></system-out>

The original use case for this, is that we have a whole suite of tests running via Jenkins, so when tests fail, it's really difficult to see what the actual problem is, as the logging is out of sync.

Issue Links

• duplicates: SUREFIRE-1643 JUnit 5 in parallel execution mode confuses Surefire reports