Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-9257

End-to-end tests prints "All tests PASS" even if individual test-script returns non-zero exit code

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 1.5.0
    • Fix Version/s: 1.6.0, 1.5.1
    • Component/s: Tests
    • Labels:
      None

      Description

      In some cases the test-suite exits with non-zero exit code but still prints "All tests PASS" to stdout. This happens because how the test runner works, which is roughly as follows

      1. Either run-nightly-tests.sh or run-precommit-tests.sh executes a suite of tests consisting of one multiple bash scripts.
      2. As soon as one of those bash scripts exists with non-zero exit code, the tests won't continue to run and the test-suite will also exit with non-zero exit code.
      3. During the cleanup hook (trap cleanup EXIT in common.sh) it will be checked whether there are non-empty out files or log files with certain exceptions. If a tests fails with non-zero exit code, but does not have any exceptions or .out files, this will still print "All tests PASS" to stdout, even though they don't

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                florianschmidt Florian Schmidt
                Reporter:
                florianschmidt Florian Schmidt
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: