Critical Description
In some cases the test-suite exits with non-zero exit code but still prints "All tests PASS" to stdout. This happens because how the test runner works, which is roughly as follows
- Either run-nightly-tests.sh or run-precommit-tests.sh executes a suite of tests consisting of one multiple bash scripts.
- As soon as one of those bash scripts exists with non-zero exit code, the tests won't continue to run and the test-suite will also exit with non-zero exit code.
- During the cleanup hook (trap cleanup EXIT in common.sh) it will be checked whether there are non-empty out files or log files with certain exceptions. If a tests fails with non-zero exit code, but does not have any exceptions or .out files, this will still print "All tests PASS" to stdout, even though they don't
Attachments
Issue Links
- links to
