We should break down the tests into categories.
- ones that ensure the test env is set up correctly for any integration tests.
- integration tests
- unit tests (ones that use RAMDirectory, in memory objects, etc)
- any others that may be useful.
The unit tests should always run. The integration tests should fail when things are not set up right.
CI should run these by categories.
The testing env should help anyone looking to verify their testing setup. e.g. The test require a temp directory as an Environment Variable. It would be helpful to put things like these into their own tests to verify the setup and help developers figure out what they are missing.