Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
2.0
-
None
Description
A large part of the effort and changes done through SCXML-213 were tested and validated by execution (part of) the SCXML IRP test cases.
The SCXML IRP, or SCXML 1.0 Implementation Report Plan provides many 'assertion' tests based on the specification, and can be used to verify that the specification can be implemented.
This is NOT a formal specification compliance testsuite, as shouldn't be regarded as such either.
But nonetheless its been extremely valuable to recognized and understand the finer and tricky details of the specification and how an implementation is expected to work.
I've found many, even rather trivial, bugs in the current implementation through these tests, and therefore I'll now add standard support for executing these as 'tests' through Maven.
The support is provided through a dedicated and single 'W3CTests' class which is executed through separate Maven profiles (w3cTests.get, w3cTests.make and w3cTests.run).
The IRP tests themselves are not checked in (although the license would allow it) but instead are dynamically downloaded by the W3CTests class (when executing 'get).
The location where these test sources are download is under a separate folder src/w3c, which I'll mark with svn:ignore, so these will not accidentally get commited either.
The current implementation of Commons SCXML however isn't (by far) specification compliant enough to succesfully execute all these (213) tests.
Instead, I'm providing an extra 'tests.xml' configuration file through which we can configure which tests are enabled/disabled and other meta-data needed to perform and track the execution status.
Currently 80 tests are passing succesfully, and some more for only either the xpath or ecmascript datamodel.
Note: the IRP tests only assume an xpath, ecmascript or "null" (minimal) datamodel.
It probably will be worthwhile to add support to these tests (within Commons SCXML) for Jexl and Groovy as well.
Most likely these will actually already be more compliant than the current ecmascript support is...