[SUREFIRE-2213] @Categories (JUnit4) are handeld differnt than JUnit's built-in Categories-Runner

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-M5, 3.0.0-M6, 3.0.0-M7, 3.0.0-M8, 3.0.0-M9, 3.0.0, 3.1.0, 3.1.2, 3.2.1, 3.2.2
• Component/s: Junit 4.x support, Maven Surefire Plugin
• Assignee: Unassigned
• Reporter: Christian Nüssgens
• Created: 17/Nov/23 14:08
• Updated: 17/Nov/23 14:08

Description

With SUREFIRE-1695 a change was introduced which modified the behavior how <groups/> and <excludeGroups/> are handled.

Assume the following Test Classes

@Category(CategoryA.class)
public abstract class AbstractTest {
}

public class TestA extends AbstractTest {
    @Test
    public void testIt() throws Exception {
        //do test
    }
}

@Category(CategoryPlayground.class)
public class PlaygroundTestA extends TestA {

    @Test
    public void testIt() throws Exception {
        // do nasty stuff like while(true)
    }
}

The built-in categories Runner behaves as follows

@RunWith(Categories.class)
@IncludeCategory({CategoryA.class})
@SuiteClasses({TestA.class, PlaygroundTestA.class})
public class TheSuite {
}

If run: only TestA is executed. The "closest" Categories declaration wins, although the Categories-annotation itself is "inheritable".

Surefire does behave differently:

<!-- .... -->
<configuration>
              <groups>CategoryA</groups>
</configuration>
<!-- .... -->

If run (mvn test) surefire will execute TestA and PlaygroundTestA (since surefire 3.0.0-M5)

We were quite surprised that this behavior changed from surefire 2.x to 3.x and is now not reflecting the way it is implemented in JUnit itself.

I'm currently not sure who is already relying on the new behavior.
Is it possible to restore the "old" behavior? Or would it be a convenient approach to make it?configurable.