Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
This is a proposal to add surefire/failsafe property called `untilFailureLoopCount`.
I often find myself in the situation where there's a random test failure happening in CI, and have to run the one or several tests continuously until it fails, maybe running them with TRACE logging.
Normally, I'd wrap the `mvn test...` call with a script that calls it in a loop, but this can be quite "slow" since each iteration is a new JVM launch...etc.
The idea behind `untilFailureLoopCount` is that you've give it a positive number, say 1000, and the test(s) would run continuously until either a failure happens or the test is executed that many times.
I've created [this randomly failing test](https://github.com/galderz/surefire-until-failure-loop-count/blob/master/src/test/java/com/acme/maven/surefire/untilfailure/UntilFailureLoopCountTest.java) and I've run it with `mvn -Dsurefire.untilFailureLoopCount=10 test` and the output would be something like:
[INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [INFO] Running com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.001 s <<< FAILURE! - in com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest [ERROR] com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure Time elapsed: 0.001 s <<< FAILURE! java.lang.AssertionError: Just messing with your testing at com.acme.maven.surefire.untilfailure.UntilFailureLoopCountTest.testEventualFailure(UntilFailureLoopCountTest.java:16)
If maintainers are interested in this, I can complete the PR with tests, documentation and support for other providers.