Uploaded image for project: 'C++ Standard Library'
  1. C++ Standard Library
  2. STDCXX-683

implement notion of expected failures in the test suite

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 4.2.0
    • Fix Version/s: 4.2.2
    • Component/s: Test Driver, Test Harness
    • Labels:
      None
    • Severity:
      Usability

      Description

      Tests (or examples) that fail for known reasons that we haven't been able to deal with should be distinguished from failures that haven't been analyzed yet. For example, an example program that fails to compile on an older target platform because of a compiler bug that we can't find a simple/elegant workaround should be flagged as such in the test results. Similarly, a test that fails one or more assertions due to compiler or libc bugs on a specific platform (or a set of platforms) that we are unable to work around should be reported as such.

      This is important in order to reduce the currently fairly large number of unexpected failures and to be able to make changes without having to worry about regressions as much.

        Attachments

        1. codes.html
          9 kB
          Martin Sebor
        2. xcodes.html
          10 kB
          Martin Sebor

          Issue Links

            Activity

              People

              • Assignee:
                sebor Martin Sebor
                Reporter:
                sebor Martin Sebor
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - 24h Original Estimate - 24h
                  24h
                  Remaining:
                  Time Spent - 50h Remaining Estimate - 20h
                  20h
                  Logged:
                  Time Spent - 50h Remaining Estimate - 20h
                  50h