Attaching patch 1a, which is the intial check-in of the new modernized compatibility test.
See the package.html for some description. It may be helpful to generate and read the Javadocs too.
The actual tests, with some modifications, have been taken from the old compatibility test. Most of the supporting code, i.e. spawning the server and the client and the combination generator, is new.
The new version is lacking the ability to run with multiple JVM versions. That feature can be added after the initial patch has been committed - given that people feel it is worth keeping. I imagine that feature would only be used in the more comprehensive test suites, and not as part of the MATS variant run in suites.All.
I intend to commit this shortly to allow for easier testing of the new test. I hope to include the MATS variant of the test in suites.All after it has been sufficiently tested and remaining issues have been addressed.
I do believe that committing the current code is better than reviewing this large chunk of code because it allows others to easily contribute by running the tests on their systems, and because this will be opt-in. The current/old compatibility test will continue to run until the community feel comfortable to do the switch.
The drawback of having two versions is that two versions have to be maintained, but there are typically few modifications to the compatibility tests (too few?). I hope that over the summer we have made enough progress to determine the final action, which will hopefully be to delete the old test and start using new one exclusively.
Patch ready for review.