[SUREFIRE-1525] Exception in a @BeforeClass method in a JUnit suite class does not fail the build if ran in parallel

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.21.0
• Fix Version/s: waiting-for-feedback
• Component/s: Junit 4.7+ (parallel) support
• Assignee: Unassigned
• Reporter: Ivan Syarov
• Created: 12/Jun/18 09:47
• Updated: 29/Mar/22 09:01

Description

I have a @BeforeClass annotated method in a JUnit test suite class, and if an exception is thrown in the method, the parallel build either succeeds if the -DfailIfNoTests parameter is set to false or fails with "No tests were executed" if the parameter is set to true. If the build is started without the parallel parameter it fails as it should.

I used a simple project setup to reproduce. I have two test classes, each with one test method and a suite class:

TestA.class

public class TestA {
    @Test
    public void test() {
        System.out.println("TestA");
    }
}

TestB.class

public class TestB {
    @Test
    public void test() {
        System.out.println("TestB");
    }
}

TestSuite.class

@RunWith(Suite.class)
@SuiteClasses({TestA.class, TestB.class})
public class TestSuite {
    @BeforeClass
    public static void setUp() {
        throw new RuntimeException("ex");
    }
}

If i execute:

mvn clean install  -Dtest=TestSuite -Dparallel=classes -DthreadCount=2 -DfailIfNoTests=false

the build succeeds. If I omit the -DfailIfNoTests=false the build fails, but not because of the thrown exception, but with "No tests were executed!". If i omit the parallel parameter the build fails appropriately with "There are test failures."

The JUnit version is 4.12

The surefire plugin version is 2.19.1 - 2.21.0