Test framework should allow tests that are expected to fail to be checked in

• Key: OPENJPA-123
• Type: Bug
• Status: Resolved
• Resolution: Fixed
• Priority: Major
• Assignee: Unassigned
• Reporter: Patrick Linskey
• Component/s: build / infrastructure
• Fix Version/s: 2.0.0-M1, 2.0.0-M2, 2.0.0-M3, 2.0.0-M4

Description

Often, tests are developed before solutions are implemented. It should be possible for us to check in such tests to svn without causing the build to fail.

Comments

[Patrick Linskey]

OPENJPA-122 is an example of a test case that should have been commitable before the behavior was implemented.

[Marc Prud'hommeaux]

I believe the Maven test plugin ("surefire") supports both JUnit and TestNG at the same time. If so, then we could use the TestNG concept of "groups" (http://testng.org/doc/documentation-main.html#test-groups) to be able to specify groups for bugs that we expect to fail.

[Dain Sundstrom]

You can list excluded tests in the maven pom. If you are using TestNG, there is a way to configure that in the pom also. See the surefire plugin documentation here:

http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html

[Patrick Linskey]

I think that I prefer moving to TestNG; I like the idea of keeping the information about whether or not the test should run colocated with the test itself, rather than separated out into the pom.

[Pinaki Poddar]

We now have multiple mechanisms to suppress a test failure.