Details

    • Type: Wish Wish
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.11
    • Component/s: Junit 4.x support
    • Labels:
      None

      Description

      Is there any plan to support using JUNIT extensions such as @Category,@PreRequisite with Maven2 SureFire plugin?
      The JUNIT EXTENSION URL:
      http://www.junitext.org/

      We would like to specify the categories to run via a configurable option in the maven surefire plugin that supports JUNIT extensions

      See example Java Code: The following runs only tests with Category - Z.

      //In JUnit4
      JUnitCore core = new JUnitCore();
      // use for categories special listener, give some statistics
      core.addListener(new CategoryTextListener(System.out));
      Request req = Request.aClass(SpcfXXXXTest.class);
      core.run(req.filterWith(new CategoryFilter("Z")));

      1. surefire-329.txt
        29 kB
        Todd Lipcon
      2. surefire-329.txt
        117 kB
        Todd Lipcon
      3. SUREFIRE329_branch329.patch
        6 kB
        Nicolas Liochon

        Issue Links

          Activity

          Anuj created issue -
          Brett Porter made changes -
          Field Original Value New Value
          Fix Version/s 2.x [ 13647 ]
          Hide
          massimo zanette added a comment -

          Does anyone know if there is a workaround to this at the moment? I am a bit surprised this issue only got 3 votes, as JUnit is quite popular. Also, Maven has poor support for functional/integration tests at the moment: if surefire had a way to harness junit categories, it would help quite a bit in organizing unit and integration tests within Maven.

          Show
          massimo zanette added a comment - Does anyone know if there is a workaround to this at the moment? I am a bit surprised this issue only got 3 votes, as JUnit is quite popular. Also, Maven has poor support for functional/integration tests at the moment: if surefire had a way to harness junit categories, it would help quite a bit in organizing unit and integration tests within Maven.
          Hide
          Matheus Cabral added a comment -

          This issue is vital to the survival of JUnit. TestNG already supports groups and JUnit now supports too. So why don't implement it?

          Show
          Matheus Cabral added a comment - This issue is vital to the survival of JUnit. TestNG already supports groups and JUnit now supports too. So why don't implement it?
          Hide
          The Alchemist added a comment -

          Anybody knows how Maven's TestNG supports groups? Does it scan the classpath or just check each file, one by one?

          Show
          The Alchemist added a comment - Anybody knows how Maven's TestNG supports groups? Does it scan the classpath or just check each file, one by one?
          Hide
          Timothy Mecklem added a comment -

          This would help us immensely to distinguish between the tests we want to run during local builds and those we want to execute during integrations which are slower. Please include this feature.

          Show
          Timothy Mecklem added a comment - This would help us immensely to distinguish between the tests we want to run during local builds and those we want to execute during integrations which are slower. Please include this feature.
          Hide
          Roy Truelove added a comment -

          This would be huge. No more having to have hacky package names to differentiate between unit/integration or small/medium/large tests.

          Show
          Roy Truelove added a comment - This would be huge. No more having to have hacky package names to differentiate between unit/integration or small/medium/large tests.
          Hide
          Todd Lipcon added a comment -

          I took a swing at implementing this. I'm new to the codebase, so apologies if this wasn't the right approach - I essentially duplicated the junit47 provider into a new JUnit48 provider, where I added the support for categories.

          I haven't done extensive testing but there is a basic integration test which passes here.

          Show
          Todd Lipcon added a comment - I took a swing at implementing this. I'm new to the codebase, so apologies if this wasn't the right approach - I essentially duplicated the junit47 provider into a new JUnit48 provider, where I added the support for categories. I haven't done extensive testing but there is a basic integration test which passes here.
          Todd Lipcon made changes -
          Attachment surefire-329.txt [ 56445 ]
          Hide
          Todd Lipcon added a comment -

          sorry, previous patch was a relative one to the copied source. This attachment is the full patch against trunk.

          Show
          Todd Lipcon added a comment - sorry, previous patch was a relative one to the copied source. This attachment is the full patch against trunk.
          Todd Lipcon made changes -
          Attachment surefire-329.txt [ 56446 ]
          Kristian Rosenvold made changes -
          Assignee Kristian Rosenvold [ krosenvold ]
          Kristian Rosenvold made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Julius made changes -
          Link This issue relates to SUREFIRE-656 [ SUREFIRE-656 ]
          Hide
          Kristian Rosenvold added a comment - - edited

          Note that I have this fully integrated into a branch at https://github.com/krosenvold/maven-surefire/tree/SUREFIRE329

          I have merged the changes back into the surefire-junit47 and extracted a common-junit48 module instead, this uses only a little bit of reflection and avoids the massive duplication in the submitted patch. I only want to test how this interacts with the parallel stuff before committing this fix to svn

          Show
          Kristian Rosenvold added a comment - - edited Note that I have this fully integrated into a branch at https://github.com/krosenvold/maven-surefire/tree/SUREFIRE329 I have merged the changes back into the surefire-junit47 and extracted a common-junit48 module instead, this uses only a little bit of reflection and avoids the massive duplication in the submitted patch. I only want to test how this interacts with the parallel stuff before committing this fix to svn
          Hide
          Todd Lipcon added a comment -

          Thanks Kristian. I had considered the reflection route as well, but it looked from the other junit versions that code duplication was the status quo. I agree reflection avoids a lot of needless duplication. Let me know if I can assist in testing. Very much looking forward to this feature!

          Show
          Todd Lipcon added a comment - Thanks Kristian. I had considered the reflection route as well, but it looked from the other junit versions that code duplication was the status quo. I agree reflection avoids a lot of needless duplication. Let me know if I can assist in testing. Very much looking forward to this feature!
          Hide
          Tibor Digana added a comment -

          Hi guys,
          I have working fix for this issue perhaps one month. How this can be provided?
          I had to update org.junit.experimental.categories.Categories.

          The user needs to pass two properties (or one of them at least) via the Maven surefire plugin namely:
          org.junit.categories.included
          org.junit.categories.excluded

          <plugin>
          <groupId>org.apache.maven.plugins</groupId>
          <artifactId>maven-surefire-plugin</artifactId>
          <version>2.6</version>
          <configuration>
          <systemPropertyVariables>
          <org.junit.categories.included>org/IRelationalDB.java</org.junit.categories.included>
          <org.junit.categories.excluded>org/IRunWithMySQL.java</org.junit.categories.excluded>
          </systemPropertyVariables>
          </configuration>
          </plugin>

          Both properties are in plural because the java classes can be separated by colon.
          Inheritance of the types in the categories are also working fine.

          Show
          Tibor Digana added a comment - Hi guys, I have working fix for this issue perhaps one month. How this can be provided? I had to update org.junit.experimental.categories.Categories. The user needs to pass two properties (or one of them at least) via the Maven surefire plugin namely: org.junit.categories.included org.junit.categories.excluded <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>2.6</version> <configuration> <systemPropertyVariables> <org.junit.categories.included>org/IRelationalDB.java</org.junit.categories.included> <org.junit.categories.excluded>org/IRunWithMySQL.java</org.junit.categories.excluded> </systemPropertyVariables> </configuration> </plugin> Both properties are in plural because the java classes can be separated by colon. Inheritance of the types in the categories are also working fine.
          Hide
          Tibor Digana added a comment -

          If we update the JUnit Categories, you don't have to fix this in surefire, because the user may use these two props.
          One way or another the junit would have to be touched anyway, even after a fix in surefire.
          IMHO no big change for a user to specify two properties in the Maven Surefire or the group.
          I have already informed the JUnit group about my fix, so i am just awaiting their answer now.
          Anyway if you fix the group in surefire then the documentation would have to be updated into new version and the backward compatibility must be guaranteed against JUnit and TestNG.

          In my fix of JUnit 4 an intersection between s/w categories and those configurable categories specified by Maven is performed to a narrow set of categories.
          I guess that the latest version 4.9.x makes only sense to integrate new changes.

          Show
          Tibor Digana added a comment - If we update the JUnit Categories, you don't have to fix this in surefire, because the user may use these two props. One way or another the junit would have to be touched anyway, even after a fix in surefire. IMHO no big change for a user to specify two properties in the Maven Surefire or the group. I have already informed the JUnit group about my fix, so i am just awaiting their answer now. Anyway if you fix the group in surefire then the documentation would have to be updated into new version and the backward compatibility must be guaranteed against JUnit and TestNG. In my fix of JUnit 4 an intersection between s/w categories and those configurable categories specified by Maven is performed to a narrow set of categories. I guess that the latest version 4.9.x makes only sense to integrate new changes.
          Hide
          Todd Lipcon added a comment -

          Kristian: any news on getting this integrated into an official SureFire release? I'm involved in two open source projects that would like to use this, but we don't want to have to build/distribute a patched surefire

          Show
          Todd Lipcon added a comment - Kristian: any news on getting this integrated into an official SureFire release? I'm involved in two open source projects that would like to use this, but we don't want to have to build/distribute a patched surefire
          Hide
          Nicolas Liochon added a comment -

          Kristian: Could you share the result of the tests you mentioned on the Aug. 29th? If the tests are ok, is it a feature you plan to integrate soon? Thank you!

          Show
          Nicolas Liochon added a comment - Kristian: Could you share the result of the tests you mentioned on the Aug. 29th? If the tests are ok, is it a feature you plan to integrate soon? Thank you!
          Hide
          Nicolas Liochon added a comment -

          I tested the branch SUREFIRE329 above on Apache HBase, mainly in forkMode=always or forkMode=once, with JUnit 4.8.2, and I have been able to make it work with some modifications on surefire. I attach the patch here if you want to reuse some elements. The issues I had are:

          1) logs on console despite a redirect setting
          2) excludedGroups was not taken into account, it seems that there is a confusion between excludedgroups & excludegroups (see the missing 'd')
          3) the filtering was not done

          I also had an issue linked to JUnit: JUnit fully initializes the classes to check if it must be included or not. It means creating the static variable & so on. If the initialization is not possible for any reason (for example complex initialization dependent on the class loader) it breaks... I created an issue in JUnit for that: https://github.com/KentBeck/junit/issues/359. It works fine with a minor modification on JUnit. We need it for HBase, it won't be necessary for everybody I think.

          Using forkMode=once, with only excludedGroups (no groups specify) does not work (junit exception). I didn't dig in this one as in our case forkMode=once without groups is barely used. It could be a pure JUnit issue.

          Show
          Nicolas Liochon added a comment - I tested the branch SUREFIRE329 above on Apache HBase, mainly in forkMode=always or forkMode=once, with JUnit 4.8.2, and I have been able to make it work with some modifications on surefire. I attach the patch here if you want to reuse some elements. The issues I had are: 1) logs on console despite a redirect setting 2) excludedGroups was not taken into account, it seems that there is a confusion between excludedgroups & excludegroups (see the missing 'd') 3) the filtering was not done I also had an issue linked to JUnit: JUnit fully initializes the classes to check if it must be included or not. It means creating the static variable & so on. If the initialization is not possible for any reason (for example complex initialization dependent on the class loader) it breaks... I created an issue in JUnit for that: https://github.com/KentBeck/junit/issues/359 . It works fine with a minor modification on JUnit. We need it for HBase, it won't be necessary for everybody I think. Using forkMode=once, with only excludedGroups (no groups specify) does not work (junit exception). I didn't dig in this one as in our case forkMode=once without groups is barely used. It could be a pure JUnit issue.
          Hide
          Nicolas Liochon added a comment -

          patch on top of the branch 329.

          Show
          Nicolas Liochon added a comment - patch on top of the branch 329.
          Nicolas Liochon made changes -
          Attachment SUREFIRE329_branch329.patch [ 57612 ]
          Hide
          Kristian Rosenvold added a comment -

          Fixed in r1199853, thanks for the patches !

          @nkeywal I only applied some of your changes; this was a big patch and it was quite complex. Your patch covered various topics and I would appreciate it if you create individual issues/patches. There are other open issues relating to stdout/stderr logging, for 2.10 so this will be fixed in that context. As for the exclude stuff, if it does not work to satisfaction, please resubmit patch with testcase. I tried to standardize on one spelling of "excludedgroups", unsure why there needs to be two different properties.

          To make a testcase you can adapt/modify the test project in surefire-integration-tests/src/test/resources/junit48-categories and add test cases to JUnit48TestCategoriesIT

          Show
          Kristian Rosenvold added a comment - Fixed in r1199853, thanks for the patches ! @nkeywal I only applied some of your changes; this was a big patch and it was quite complex. Your patch covered various topics and I would appreciate it if you create individual issues/patches. There are other open issues relating to stdout/stderr logging, for 2.10 so this will be fixed in that context. As for the exclude stuff, if it does not work to satisfaction, please resubmit patch with testcase. I tried to standardize on one spelling of "excludedgroups", unsure why there needs to be two different properties. To make a testcase you can adapt/modify the test project in surefire-integration-tests/src/test/resources/junit48-categories and add test cases to JUnit48TestCategoriesIT
          Kristian Rosenvold made changes -
          Resolution Fixed [ 1 ]
          Status In Progress [ 3 ] Closed [ 6 ]
          Hide
          Nicolas Liochon added a comment -

          Hi Kristian, thanks for having integrated the functionality. It's really great. My tests and changes were driven a lot by the forkMode=always. I am going to check the version on trunk and I will create some specific jira if I find something.

          Show
          Nicolas Liochon added a comment - Hi Kristian, thanks for having integrated the functionality. It's really great. My tests and changes were driven a lot by the forkMode=always. I am going to check the version on trunk and I will create some specific jira if I find something.
          Nicolas Liochon made changes -
          Link This issue is related to SUREFIRE-786 [ SUREFIRE-786 ]
          Kristian Rosenvold made changes -
          Fix Version/s Backlog [ 13647 ]
          Fix Version/s 2.11 [ 17856 ]
          Mark Thomas made changes -
          Project Import Sun Apr 05 13:23:32 UTC 2015 [ 1428240212200 ]
          Mark Thomas made changes -
          Workflow jira [ 12727581 ] Default workflow, editable Closed status [ 12758656 ]
          Mark Thomas made changes -
          Project Import Mon Apr 06 01:36:33 UTC 2015 [ 1428284193036 ]
          Mark Thomas made changes -
          Workflow jira [ 12965558 ] Default workflow, editable Closed status [ 13003250 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          1559d 13h 6m 1 Kristian Rosenvold 24/Aug/11 03:40
          In Progress In Progress Closed Closed
          77d 8h 28m 1 Kristian Rosenvold 09/Nov/11 11:09

            People

            • Assignee:
              Kristian Rosenvold
              Reporter:
              Anuj
            • Votes:
              37 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development