[NETBEANS-5863] "Test results" spuriously clears tests during run, and shows success despite failure

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 12.4
• Fix Version/s: 12.5
• Component/s: java - JUnit, utilities - Test Runner
• Labels: pull-request-available

Environment

NetBeans 12.4 on 64-bit Windows 10 on OpenJDK 11.0.11.

• Assignee: Dusan Balek
• Reporter: Eirik Bakke
• Created: 18/Jul/21 12:22
• Updated: 26/Jul/21 13:53
• Resolved: 23/Jul/21 15:46

Description

In NetBeans 12.4, test results displayed in the "Test results" pane will get randomly cleared while the tests are running. The final output thus ends up not including all tests that were run. See the attached series of screenshots. Sometimes the "Test results" pane will end up showing a green bar with "Tests passed: 100.00 %" even when there were test failures.

My application's test suite is JUnit in a maven-based NetBeans module running maven-surefire-plugin version 3.0.0-M4. Using an older version of the surefire plugin (2.18.1, which I previously used to avoid NETBEANS-458 ) did not solve the problem.

This problem occurs on NetBeans 12.4, but not on NetBeans 12.1. I verified this by running my application's test suite on both NetBeans versions, without changing surefire versions or anything else.

Issue Links

• links to: GitHub Pull Request #3072