Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: None
    • Labels:
      None

      Description

      As you probably know, the JUnit team is currently working on a new major version, codenamed JUnit Lambda. We are still in a very early prototype/alpha phase but nevertheless wanted to reach out to you regarding future support for JUnit 5 in Maven Surefire.

      The prototype

      We have even gone so far as to create our own SurefireProvider for the prototype we've release a couple of weeks ago: https://github.com/junit-team/junit-lambda/tree/prototype-1/surefire-junit5. It's obviously very basic and only supports the most common use case of running all test classes in a module. This was merely meant as a proof of concept. Moreover, we do not plan to release and maintain our own SurefireProvider in the long run. That's why I'm opening this issue so we start to talk about it and then, hopefully, start collaborating on it.

      Things the JUnit team has decided

      • JUnit 5 will require a JDK 8 runtime
      • JUnit 5 will be able to run JUnit 4 and JUnit 3 tests if the right JARs happen to be on the classpath.

      Possible next steps

      We would love if you could take a look at our current build tool/IDE facing API if you have the time. Please feel free to ask any question that might come up.

      We are currently working on the first alpha. Our goal is to have a stable build tool/IDE facing API by then but as sooner as you provide feedback the better.

        Activity

        Hide
        tibor17 Tibor Digana added a comment -

        Marc Philipp
        Can you guarantee that this code will recognize all JUnit runners embedded in junit5 library?
        We don't want to make any lookup in junit in order to instantiate Runner upon RunWith annotation.

        Computer computer = createComputer();
        Request req = Request.classes( computer, classesToRun );
        req.getRunner().run( notifier );
        

        When do you plan the release day in junit5?

        Currently all test classes, methods, fields must be public.
        How about in junit5?

        Is or will be RunListener,Notifier,Result,Request,Filter,Runner,ParentRunner method signatures updated in junit5?

        Are the Assertion and Assumption exceptions backwards compatible and are their signatures unmodified in junit5? We still rely on junit4.

        At the moment I see only tagging [1] and ignored tests [2] to be implemented in Surefire.
        [1] https://github.com/junit-team/junit-lambda/wiki/Prototype-Writing-Test-Cases#tagging-and-filtering
        [2] https://github.com/junit-team/junit-lambda/wiki/Prototype-Writing-Test-Cases#disabling-tests
        Any patches are welcome.

        It would nice to run ITs with junit5.

        Show
        tibor17 Tibor Digana added a comment - Marc Philipp Can you guarantee that this code will recognize all JUnit runners embedded in junit5 library? We don't want to make any lookup in junit in order to instantiate Runner upon RunWith annotation. Computer computer = createComputer(); Request req = Request.classes( computer, classesToRun ); req.getRunner().run( notifier ); When do you plan the release day in junit5? Currently all test classes, methods, fields must be public. How about in junit5? Is or will be RunListener,Notifier,Result,Request,Filter,Runner,ParentRunner method signatures updated in junit5? Are the Assertion and Assumption exceptions backwards compatible and are their signatures unmodified in junit5? We still rely on junit4. At the moment I see only tagging [1] and ignored tests [2] to be implemented in Surefire. [1] https://github.com/junit-team/junit-lambda/wiki/Prototype-Writing-Test-Cases#tagging-and-filtering [2] https://github.com/junit-team/junit-lambda/wiki/Prototype-Writing-Test-Cases#disabling-tests Any patches are welcome. It would nice to run ITs with junit5.
        Hide
        marcphilipp Marc Philipp added a comment -

        Is or will be RunListener,Notifier,Result,Request,Filter,Runner,ParentRunner method signatures updated in junit5?

        JUnit 5 will be a completely new framework, not an evolution of the JUnit 4 codebase. We will not modify any classes or signatures in JUnit 4 code. Contrary to what JUnit 4 did with JUnit 3, JUnit 5 will not include JUnit 4 in its JAR. JUnit 5 will not use JUnit 4 annotations but provide its own.

        In order to run JUnit 4 tests and JUnit 5 tests in the same run, JUnit 5 will introduce a new abstraction: The Launcher and Engine API. The Launcher is the API that build tools and IDEs will use to specify which tests the framework should run. It has a new listener (TestExecutionListener) that you will have to implement. The Launcher can discover and execute tests on multiple Engines in a single run. The Engines register themselves with the Launcher using the JDK's ServiceLoader. We will provide engines for JUnit 5 and JUnit 4. If both engines are on the classpath, the Launcher will be able to execute JUnit 4 and JUnit 5 tests in a single run. The JUnit 4 Engine will depend on the JUnit 4 JAR as it is today. The JUnit 5 engine will be able to run tests that use a new API.

        You can think of the Launcher as a replacement for JUnitCore, TestExecutionListener replaces RunListener, an Engine is in some ways similar to a Runner. Please take a look at the SurefireProvider prototype I've linked to in the ticket description.

        Can you guarantee that this code will recognize all JUnit runners embedded in junit5 library?
        We don't want to make any lookup in junit in order to instantiate Runner upon RunWith annotation.

        You will not have to make any lookup to get the Runner for a class. The JUnit 4 engine will do that, just like JUnitCore does now.

        When do you plan the release day in junit5?

        We plan to publish a first alpha release in February 2016. We encourage you to provide feedback regarding the Launcher and Engine API. We want them to be stable by then. We want to ship a release candidate in Summer/Autumn 2016.

        Currently all test classes, methods, fields must be public.
        How about in junit5?

        In JUnit 5 they don't have to be public.

        Are the Assertion and Assumption exceptions backwards compatible and are their signatures unmodified in junit5? We still rely on junit4.

        JUnit 5 will use different exceptions, and change a few of the signatures of assertion/assumption methods. However, you can migrate step by step and continue using JUnit 4 Assert because JUnit 5 will provide a new class in a new package (org.junit.gen5.api.Assertions).

        At the moment I see only tagging and ignored tests to be implemented in Surefire.

        For tagging we can use Surefire's groups and ignored tests will have to be reported in the same way as now, right?

        Show
        marcphilipp Marc Philipp added a comment - Is or will be RunListener,Notifier,Result,Request,Filter,Runner,ParentRunner method signatures updated in junit5? JUnit 5 will be a completely new framework, not an evolution of the JUnit 4 codebase. We will not modify any classes or signatures in JUnit 4 code. Contrary to what JUnit 4 did with JUnit 3, JUnit 5 will not include JUnit 4 in its JAR. JUnit 5 will not use JUnit 4 annotations but provide its own. In order to run JUnit 4 tests and JUnit 5 tests in the same run, JUnit 5 will introduce a new abstraction: The Launcher and Engine API. The Launcher is the API that build tools and IDEs will use to specify which tests the framework should run. It has a new listener (TestExecutionListener) that you will have to implement. The Launcher can discover and execute tests on multiple Engines in a single run. The Engines register themselves with the Launcher using the JDK's ServiceLoader. We will provide engines for JUnit 5 and JUnit 4. If both engines are on the classpath, the Launcher will be able to execute JUnit 4 and JUnit 5 tests in a single run. The JUnit 4 Engine will depend on the JUnit 4 JAR as it is today. The JUnit 5 engine will be able to run tests that use a new API. You can think of the Launcher as a replacement for JUnitCore, TestExecutionListener replaces RunListener, an Engine is in some ways similar to a Runner. Please take a look at the SurefireProvider prototype I've linked to in the ticket description. Can you guarantee that this code will recognize all JUnit runners embedded in junit5 library? We don't want to make any lookup in junit in order to instantiate Runner upon RunWith annotation. You will not have to make any lookup to get the Runner for a class. The JUnit 4 engine will do that, just like JUnitCore does now. When do you plan the release day in junit5? We plan to publish a first alpha release in February 2016. We encourage you to provide feedback regarding the Launcher and Engine API. We want them to be stable by then. We want to ship a release candidate in Summer/Autumn 2016. Currently all test classes, methods, fields must be public. How about in junit5? In JUnit 5 they don't have to be public. Are the Assertion and Assumption exceptions backwards compatible and are their signatures unmodified in junit5? We still rely on junit4. JUnit 5 will use different exceptions, and change a few of the signatures of assertion/assumption methods. However, you can migrate step by step and continue using JUnit 4 Assert because JUnit 5 will provide a new class in a new package (org.junit.gen5.api.Assertions). At the moment I see only tagging and ignored tests to be implemented in Surefire. For tagging we can use Surefire's groups and ignored tests will have to be reported in the same way as now, right?
        Hide
        tibor17 Tibor Digana added a comment -

        >>For tagging we can use Surefire's groups and ignored tests will have to be reported in the same way as now, right?
        Groups and tags seem to be compliant.

        Show
        tibor17 Tibor Digana added a comment - >>For tagging we can use Surefire's groups and ignored tests will have to be reported in the same way as now, right? Groups and tags seem to be compliant.

          People

          • Assignee:
            Unassigned
            Reporter:
            marcphilipp Marc Philipp
          • Votes:
            2 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:

              Development