Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-505

Convert the file including 10 minutes run's tests into the test harness test suite

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: build, test
    • Labels:
      None

      Description

      This issue is created to track the conversion of external testlist for 10 minutes tests into JUnit test suite.

      It would be consistent through the bigger Hadoop project to have 10 minutes tests to be collected in a JUnit's test suite instead of an external file (as of now).

      The reason is simple: with the file we'd have two points of maintenance: tests source code and one an auxiliary text files. Besides, a general approach is to use JUnit's suites for tagging purposes, thus having a suite here would be more uniform.

        Issue Links

          Activity

          Hide
          cos Konstantin Boudnik added a comment -

          The conversation about this conversion has been already carried in HDFS-458

          Show
          cos Konstantin Boudnik added a comment - The conversation about this conversion has been already carried in HDFS-458
          Hide
          cos Konstantin Boudnik added a comment -

          In order to clear possible confusion, I'd like to add a couple of comments here:

          • it's big shame that JUnit doesn't support tests tagging contrary to a popular believe (see my comments for HADOOP-4901)
          • because there's some opposition to bring in a 3rd party libraries into the project (again see HADOOP-4901) the tagging would have to be implemented through JUnit test suites instead.
          • apparently, as Jacob has pointed out - we'd need to work around a potential problem of running the same tests twice (10 minutes ones and all of them in a single run). And there are some possible solutions for this, e.g. setting up test.exclude property to remove 10 minutes suite from a normal run; or to have a certain test class naming conventions, i.e. add a token 'Suite' into a class name and filter such classes out in a normal runs. Other approaches are, perhaps, possible to.
          Show
          cos Konstantin Boudnik added a comment - In order to clear possible confusion, I'd like to add a couple of comments here: it's big shame that JUnit doesn't support tests tagging contrary to a popular believe (see my comments for HADOOP-4901 ) because there's some opposition to bring in a 3rd party libraries into the project (again see HADOOP-4901 ) the tagging would have to be implemented through JUnit test suites instead. apparently, as Jacob has pointed out - we'd need to work around a potential problem of running the same tests twice (10 minutes ones and all of them in a single run). And there are some possible solutions for this, e.g. setting up test.exclude property to remove 10 minutes suite from a normal run; or to have a certain test class naming conventions, i.e. add a token 'Suite' into a class name and filter such classes out in a normal runs. Other approaches are, perhaps, possible to.
          Hide
          jghoman Jakob Homan added a comment -

          In my opinion, having the 10-minute-tests defined in an external file is a feature, not a bug. It makes it easy to quickly see what's in the suite quickly and easily update it without touching code. The set membership seems to be a matter of configuration, which should go into a separate file, rather than code, which should go into a test suite. Tagging had seemed like a good solution because a test's membership into the set can be viewed as metadata, but if we don't really have annotation-type tagging, it may not be worth it.
          I had hoped to use tagging because I wanted to tag individual tests from a test suite rather than running all the tests in the suite, as we have to do now. But it sounds like that's not possible, even creating a new test suite. Or am I wrong about that?
          If it's not possible to include individual tests into the 10min suite via annotation-type tagging, and if we're going to have to do something tricky/clever to get the test suite approach to work (as Cos mentions above), I'd rather keep the flat files.

          Show
          jghoman Jakob Homan added a comment - In my opinion, having the 10-minute-tests defined in an external file is a feature, not a bug. It makes it easy to quickly see what's in the suite quickly and easily update it without touching code. The set membership seems to be a matter of configuration, which should go into a separate file, rather than code, which should go into a test suite. Tagging had seemed like a good solution because a test's membership into the set can be viewed as metadata, but if we don't really have annotation-type tagging, it may not be worth it. I had hoped to use tagging because I wanted to tag individual tests from a test suite rather than running all the tests in the suite, as we have to do now. But it sounds like that's not possible, even creating a new test suite. Or am I wrong about that? If it's not possible to include individual tests into the 10min suite via annotation-type tagging, and if we're going to have to do something tricky/clever to get the test suite approach to work (as Cos mentions above), I'd rather keep the flat files.
          Hide
          cos Konstantin Boudnik added a comment -

          It is a feature indeed, that's why this JIRA has been opened as an improvement rather than a bug.

          I'm totally agree that tagging is a way MORE superior solution to a haness' suite constructions. One of the best features of tagging solutions I have proposed in HADOOP-4901 is an ability to tag a single test case which is very difficult to do with JUnit's built in Suites. I'm adding Owen to the watcher's list for he had a very strong opinion of why JUnit suites have to be used instead of say Spring-test framework annotations.

          This whole 10 minutes test list has exposed the problem with suites vs. metadata vs. external test lists. I'm Ok with external test lists but for two reasons:

          • they require an extra maintenance step is needed
          • at this moment, HDFS seems to be the only Hadoop's project which uses test lists. Thus it creates a potential future inconsistency.
          Show
          cos Konstantin Boudnik added a comment - It is a feature indeed, that's why this JIRA has been opened as an improvement rather than a bug. I'm totally agree that tagging is a way MORE superior solution to a haness' suite constructions. One of the best features of tagging solutions I have proposed in HADOOP-4901 is an ability to tag a single test case which is very difficult to do with JUnit's built in Suites. I'm adding Owen to the watcher's list for he had a very strong opinion of why JUnit suites have to be used instead of say Spring-test framework annotations. This whole 10 minutes test list has exposed the problem with suites vs. metadata vs. external test lists. I'm Ok with external test lists but for two reasons: they require an extra maintenance step is needed at this moment, HDFS seems to be the only Hadoop's project which uses test lists. Thus it creates a potential future inconsistency.
          Hide
          cos Konstantin Boudnik added a comment -

          Please see my comment on MAPREDUCE-670. Generally, I tend to close this JIRA because JUnit test suite won't provide us with any additional benefits compare to a flat test list.

          Show
          cos Konstantin Boudnik added a comment - Please see my comment on MAPREDUCE-670 . Generally, I tend to close this JIRA because JUnit test suite won't provide us with any additional benefits compare to a flat test list.

            People

            • Assignee:
              Unassigned
              Reporter:
              cos Konstantin Boudnik
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development