Uploaded image for project: 'TinkerPop'
  1. TinkerPop
  2. TINKERPOP-2013

Process tests that are auto-ignored stink

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 3.2.9
    • Fix Version/s: 3.4.0, 3.3.5
    • Component/s: test-suite
    • Labels:
      None

      Description

      I think I've had it with GremlinProcessRunner and any other related infrastructure that auto-ignores batches of tests unknowingly. The typical scenario is that a new test gets added, we build the project and everything works without failures. Then something starts misbehaving and we wonder why. After a lot of cycles and looking in the wrong places we realize that the tests that should have caught the problem are simply being ignored.

      There was a reason for making things work this way as we had assumed that certain tests failing in a certain way was expected behavior, but that is not always the case. We’ve learned that assumption doesn’t always apply and it would be better to be more explicit in dealing with those failures.

      Not sure how this is going to be resolved, but I think that something needs to change. Perhaps we need to identify all the auto-ignored tests and do any of the following:

      1. Make them work
      2. Explicitly ignore them
      3. Re-organize tests so that it's not assumed that they just are part of the process suite
      4. Something else??

      May also need to extend this into all the ignored GLV tests that aren't supported for various reasons. Perhaps the assertion language needs more expressiveness.

        Attachments

          Activity

            People

            • Assignee:
              spmallette Stephen Mallette
              Reporter:
              spmallette Stephen Mallette

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment