[JDO-765] TCK should fail if tests fail

• Type: Bug
• Status: Closed
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: JDO 3.1
• Fix Version/s: JDO 3.2
• Component/s: tck
• Assignee: Tilmann Zäschke
• Reporter: Tilmann Zäschke
• Created: 04/Nov/17 17:14
• Updated: 17/Feb/22 19:54
• Resolved: 07/Dec/17 17:14

Description

Currently, TCK test failures are reported to the console and in the logs, but the Maven build still succeeds.

The proposal is to make the behavior configurable, the default being that the Maven build fails if any TCK tests fail.

The new configuration option would be called "-Djdo.tck.onFailure" and have the following options:

• "failFast" will immediately abort the test run.
• "failEventually" (default) will execute the whole TCK before failing.
• "logOnly" will report failures to console and logs only but return 'SUCCESS' to the Maven execution environment.

Attached is a patch with an implementation of the proposed changes.