Details

    • Type: Task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: Lucene.Net 3.0.3, Lucene.Net 4.8.0
    • Fix Version/s: None
    • Labels:
    • Environment:

      All

      Description

      We should break down the tests into categories.

      Possible Categories:

      • ones that ensure the test env is set up correctly for any integration tests.
      • integration tests
      • unit tests (ones that use RAMDirectory, in memory objects, etc)
      • any others that may be useful.

      The unit tests should always run. The integration tests should fail when things are not set up right.

      CI should run these by categories.

      The testing env should help anyone looking to verify their testing setup. e.g. The test require a temp directory as an Environment Variable. It would be helpful to put things like these into their own tests to verify the setup and help developers figure out what they are missing.

        Activity

        Hide
        NightOwl888 Shad Storhaug added a comment -

        I am not sure I understand the problem that would be solved by making these changes.

        The unit tests should always run. The integration tests should fail when things are not set up right.

        The testing env should help anyone looking to verify their testing setup. e.g. The test require a temp directory as an Environment Variable. It would be helpful to put things like these into their own tests to verify the setup and help developers figure out what they are missing.

        In Lucene.Net 4.8.0, there are no requirements to have any environment variable setup for the tests to run. As long as all of the prerequisites are in place (as specified on the readme https://github.com/apache/lucenenet#building-and-testing), the tests will run.

        Am I correct in assuming that this request was a workaround for the fact that Lucene.Net 3.0.3 didn't have the tests setup to "just run" (as they should always be setup)?

        We should break down the tests into categories.

        Possible Categories:

        • ones that ensure the test env is set up correctly for any integration tests.
        • integration tests
        • unit tests (ones that use RAMDirectory, in memory objects, etc)
        • any others that may be useful.

        CI should run these by categories.

        As for categorizing tests beyond "long running tests" and "lucene.net specific tests" which we already have, I am not seeing the advantage there either. Nearly all of the tests are integration tests (that is, tests that are setup to test things as a unit instead of one individual component and a bunch of fakes). Putting 80-90% of the 7000+ tests in one big category doesn't seem like it would gain anything. Also, I am not seeing what we gain by running the categories separately in CI. Am I missing something?

        Show
        NightOwl888 Shad Storhaug added a comment - I am not sure I understand the problem that would be solved by making these changes. The unit tests should always run. The integration tests should fail when things are not set up right. The testing env should help anyone looking to verify their testing setup. e.g. The test require a temp directory as an Environment Variable. It would be helpful to put things like these into their own tests to verify the setup and help developers figure out what they are missing. In Lucene.Net 4.8.0, there are no requirements to have any environment variable setup for the tests to run. As long as all of the prerequisites are in place (as specified on the readme https://github.com/apache/lucenenet#building-and-testing ), the tests will run. Am I correct in assuming that this request was a workaround for the fact that Lucene.Net 3.0.3 didn't have the tests setup to "just run" (as they should always be setup)? We should break down the tests into categories. Possible Categories: ones that ensure the test env is set up correctly for any integration tests. integration tests unit tests (ones that use RAMDirectory, in memory objects, etc) any others that may be useful. CI should run these by categories. As for categorizing tests beyond "long running tests" and "lucene.net specific tests" which we already have, I am not seeing the advantage there either. Nearly all of the tests are integration tests (that is, tests that are setup to test things as a unit instead of one individual component and a bunch of fakes). Putting 80-90% of the 7000+ tests in one big category doesn't seem like it would gain anything. Also, I am not seeing what we gain by running the categories separately in CI. Am I missing something?

          People

          • Assignee:
            michaelherndon michael herndon
            Reporter:
            michaelherndon michael herndon
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Time Tracking

              Estimated:
              Original Estimate - 40h
              40h
              Remaining:
              Remaining Estimate - 40h
              40h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development