Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.18
    • Component/s: None
    • Labels:
      None

      Description

      Pull Request 40:
      https://github.com/apache/maven-surefire/pull/40

      Add rerunFailingTestsCount option for maven surefire to rerun failing tests immediately after they fail.

      When rerunFailingTestsCount is set to a value k larger than 0, a failing test will get re-run
      up to k times until it passes. If a test passes in any of its reruns,
      the build will be marked as successful and the test will count as a
      flake (or flaky test). If it fails all those k times then it will
      still be marked as a failed test.

      In the console output all the flaky tests will be count as "Flakes:
      N". The generated test report XML file is augmented with additional
      information, while still being compatible with existing consumers
      (such as Jenkins). A flaky test will have <flakyFailure> or/and
      <flakyError> under its <testcase> element, to store all the flaky
      runs' information (such as output, stackTrace). So existing consumers
      will still consider it as a passing test, while potential future
      consumers can parse those flaky runs information. A failing test will
      still have <failure> or <error> under <testcase>, but all the
      subsequent re-run information will be stored under <rerunFailure> or
      <rerunError>. So existing consumers will still be able to see it's a
      failed test and parse its failure information, and potential future
      consumers will be able to get all the flaky runs.

      It is implemented by keeping a map between test full class name and a
      map between all its test methods and the list of runs. It also takes
      into account Fork and Parallel and have them covered by integration
      tests.

      Currently only supports JUnit4.x

        Attachments

          Activity

            People

            • Assignee:
              agudian Andreas Gudian
              Reporter:
              seriousamlqz Qingzhou Luo
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: