[SUREFIRE-1860] extend ReportEntry interface and SimpleReportEntry with mandatory properties runMode:String, testRunId:long

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0-M6
• Component/s: JUnit 3.x support, Junit 4.7+ (parallel) support, Junit 4.x support, JUnit 5.x support, Maven Failsafe Plugin, Maven Surefire Plugin, TestNG support
• Assignee: Tibor Digana
• Reporter: Tibor Digana
• Created: 18/Nov/20 11:47
• Updated: 26/Mar/22 19:59
• Resolved: 06/Mar/22 11:25

Description

We are missing two properties in the ReportEntry interface

• runMode:RunMode, which "informs" the reporters that the statistics data is for normal tests or re-run tests
• testRunId:long, which identifies the test run instead of the current ID represented by the combination of class+method.

Currently the XML reporter is stateful and the run mode is determined by the timing and the order of normal tests and rerun tests. The problem is when we run the tests in parallel. After we would employ the testRunId:long we would not have these problems and we would solve much more like JUnit5's Dynamic tests and the ability to run Cucumber scripts.

The reporters should identify the test run by the combination of forkId + test run id. The forks have no notion about the other forks, and it's even more difficult to make the test ids coherent (without duplicates) across the forks because they are very dynamically created and finished during the lifetime of the plugin execution. Therefore the reporter implementation should respect the forkId too.

Currently the JUnit47 provider has a cache, TestMethod, which has the console logs in memory and sends these after the class has finished. This is memory resources problem. We are aiming for sending these logs immediately, and have the implementation stateless.

Issue Links

• is a parent of: SUREFIRE-2009 Refactoring of surefire-junit3. JUnitTestSetExecutor and PojoTestSetExecutor should be stateless.
• is a parent of: SUREFIRE-2011 Updated abstractions which helps associating standard out/err with a test
• is a parent of: SUREFIRE-2014 Implement testRunId and RunMode in the EventEncoder and EventDecoder
• is a parent of: SUREFIRE-2015 Implement testRunId and RunMode in the SimpleReportEntry
• is a parent of: SUREFIRE-2025 Updated abstractions which helps associating systemProperties() with a test context
• is a parent of: SUREFIRE-2046 Resolved TODOs. Updated callbacks ForkedProcessPropertyEventListener and ForkedProcessStandardOutErrEventListener.
• links to: GitHub Pull Request #345