[SUREFIRE-2102] Different Behaviour while using junit-jupiter-engine/jupiter-vintage-engine/testng-engine

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.0.0-M7
• Component/s: Junit 4.x support, JUnit 5.x support, TestNG support
• Environment: JDK 17 ; Maven 3.8.6 ; Surefire 3.0.0-M7
• Assignee: Unassigned
• Reporter: Karl Heinz Marbaise
• Created: 20/Jun/22 01:45
• Updated: 30/May/23 18:11

Description

I have a full working example which executes three different kind of tests:

• Junit Jupiter based
• JUnit 4 based
• TestNG based

what I can observe there seemed to be a difference in behaviour.

[INFO] Running com.soebes.youtube.maven.episodes.ep3.jupiter.FractionJUnitJupiterTest
FractionJUnitJupiterTest.beforeAll
FractionJUnitJupiterTest.beforeEach
FractionJUnitJupiterTest.first_add
FractionJUnitJupiterTest.afterEach
FractionJUnitJupiterTest.afterAll
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 s - in com.soebes.youtube.maven.episodes.ep3.jupiter.FractionJUnitJupiterTest
[INFO] Running com.soebes.youtube.maven.episodes.ep3.junit.FractionJUnit4Test
FractionJUnit4Test.beforeClass
FractionJUnit4Test.before
FractionJUnit4Test.first_add
FractionJUnit4Test.after
FractionJUnit4Test.afterClass
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in com.soebes.youtube.maven.episodes.ep3.junit.FractionJUnit4Test
[INFO] Running com.soebes.youtube.maven.episodes.ep3.testng.FractionTestNGTest
FractionTestNGTest.beforeClass
FractionTestNGTest.beforeMethod
FractionTestNGTest.first_add
FractionTestNGTest.afterMethod
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 s - in com.soebes.youtube.maven.episodes.ep3.testng.FractionTestNGTest
FractionTestNGTest.afterClass
[DEBUG] Closing the fork 1 after saying GoodBye.

You can see that the output of the afterClass is after the output of the test result. But only for TestNG while for JUnit 4 and JUnit Jupiter it looks the same.

The same happens for using failsafe in the same example project.

I made a full working reproducible example: https://github.com/khmarbaise/sffs-bugs (Branch: main). This requires JDK 17 (but should also work with JDK11)...

 The repository contains two full DEBUG logoutput files: mvn.log (running mvn verify) and mvn-test.log only for running unit test via mvn test