[SUREFIRE-1857] JUnit 5 report does not contain assertion failure message

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-M4, 3.0.0-M5
• Fix Version/s: 3.0.0-M6
• Component/s: JUnit 5.x support
• Labels: junit5
• Assignee: Tibor Digana
• Reporter: Niklas Krieger
• Created: 27/Oct/20 08:57
• Updated: 18/Nov/20 09:15
• Resolved: 18/Nov/20 08:32

Description

Since JUnit 5, if a test fails due to an assertion, the xml report does not contain the assertion message. According to the report specification, the message should be included:

<xs:element name="failure" nillable="true" minOccurs="0" maxOccurs="unbounded">
    <xs:complexType>
        <xs:simpleContent>
            <xs:extension base="xs:string">
                <xs:attribute name="message" type="xs:string"/>
                <xs:attribute name="type" type="xs:string" use="required"/>
            </xs:extension>
        </xs:simpleContent>
    </xs:complexType>
</xs:element>

 

Pom contains:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>3.0.0-M5</version>
        </plugin>
    </plugins>
</build>

Test:

package com.example;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;

public class MyTest {
    @Test
    public void test() {
        Assertions.assertEquals(1, 2, "two is not one");
    }
}

Generated report:

<testcase name="test" classname="com.example.MyTest" time="0.018">
    <failure type="org.opentest4j.AssertionFailedError"><![CDATA[org.opentest4j.AssertionFailedError: five is not six ==> expected: <5> but was: <6>
   at TestProject/com.example.MyTest.test(MyTest.java:9)
]]></failure>
</testcase>

 

Possible reason for this behavior:

In version m3, RunListenerAdapter uses SimpleReportEntry.withException to generate the entry if the test failed due to an assertion

Since Version m4, a constructor of SimpleReportEntry is used, however it gets null as message

Issue Links

• links to: GitHub Pull Request #324