[SUREFIRE-2117] XML report omits package and outer class name for tests in @Nested inner class if testset reporter is configured to use phrased naming

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-M7
• Fix Version/s: 3.0.0-M8
• Component/s: JUnit 5.x support, xml generation
• Assignee: Slawomir Jaranowski
• Reporter: Andreas Pabst
• Created: 03/Oct/22 16:07
• Updated: 27/Jan/23 07:33
• Resolved: 28/Dec/22 16:35

Description

Description of the issue

Given a test class with tests in nested inner classes as follows:

package mypackage;

class OuterTestClass {

    @Nested
    class A {

        @Test
        void level1_test() { ... }

        @Nested
        class B {

            @Test
            void level2_test() { ... }

        }
    }
}

and a surefire configuration that includes the following

<statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
    <version>3.0</version>
    <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
    <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
    <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
</statelessTestsetReporter>

the XML reports will look like this:

<testcase name="level1_test" classname="A" time="0"/>

<testcase name="level2_test" classname="B" time="0"/>

i.e. the package name, outer class name and higher level inner class names are not included in the classname attribute.

I would expect something more like this:

<testcase name="level1_test" classname="mypackage.OuterTestClass$A" time="0"/>

<testcase name="level2_test" classname="mypackage.OuterTestClass$A$B" time="0"/>

This happens also if  @DisplayName  is used by the way. Only the display name of the leaf inner class is included in the report in that case.

Some context on why this is bad

Any software that parses the surefire XML reports is missing out on potentially important context information.

Solution ideas

The root cause is similar to SUREFIRE-1914. Therefore, a similar fix can be provided: the hierarchy of class source test identifiers needs to be taken into account to determine the classname attribute.

I will provide a pull request that contains ITs that reproduce the issue and a proposed fix.

Workaround

A workaround is to explicitly specify a @DisplayName that also contains the information otherwise provided by the surrounding inner class(es), the outer class and the package name.

Issue Links

• is related to: SUREFIRE-2147 JUnit5 DynamicContainer/DynamicTest report loses the hierarchical structure
• relates to: SUREFIRE-1914 XML report omits method signature / display name of Junit 5 parameterized tests if testset reporter is configured to use phrased naming
• links to: GitHub Pull Request #568