Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.12.3
    • Fix Version/s: None
    • Component/s: test
    • Labels:
      None
    • Release Note:
      This is a work in progress.

      Description

      It would be nice to have a distributed junit runner that would run 100 instances of each test a separate map, and use counters to count how many times each test passed/failed/hung.

        Issue Links

          Activity

          Hide
          Doug Cutting added a comment -

          Bootstrapping could be a little complicated here: do we use an older version of Hadoop to test the new version? Since Hudson runs on a 4-processor box, we might get some mileage by simply getting junit tests to run in parallel. Ant's parallel task is almost appropriate, except there's no easy way I can see to get each junit test to be a sub-task of a parallel task. But perhaps a simple parallel-junit task could be written for this. Test could still be run in separate JVMs, and we could run as many at once as there are CPUs (a feature of the parallel task). A 4x speedup would be very welcome!

          Show
          Doug Cutting added a comment - Bootstrapping could be a little complicated here: do we use an older version of Hadoop to test the new version? Since Hudson runs on a 4-processor box, we might get some mileage by simply getting junit tests to run in parallel. Ant's parallel task is almost appropriate, except there's no easy way I can see to get each junit test to be a sub-task of a parallel task. But perhaps a simple parallel-junit task could be written for this. Test could still be run in separate JVMs, and we could run as many at once as there are CPUs (a feature of the parallel task). A 4x speedup would be very welcome!
          Hide
          Tom White added a comment -

          > Tom, haven't you done something similar with your https://
          > computefarm.dev.java.net/ project?

          Yes, this was some time ago now, but the basic idea was to extend JUnit (this in itself was hard since JUnit 3 was not really written for extensibility to distribute test cases to worker machines using JavaSpaces. The idea was to distribute them in order to get the whole suite to run faster, not to run each test a number of times in order to examine failure rates.

          One of the nice things was that you didn't have to install your tests on the workers since Jini code mobility ensured the bytecodes were downloaded as necessary. However you did need to make sure your tests were Serializable... More at https://computefarm.dev.java.net/samples.html and https://computefarm.dev.java.net/source/browse/computefarm/trunk/samples/src/java/org/tiling/computefarm/samples/junit/.

          I think this would be a nice general application of Hadoop, and we'd be able to avoid the restriction on tests having to be Serializable.

          However, for the immediate problem of reducing test execution time on Hudson, Doug's suggestion of writing a custom ant task sounds like it would be more appropriate.

          Show
          Tom White added a comment - > Tom, haven't you done something similar with your https:// > computefarm.dev.java.net/ project? Yes, this was some time ago now, but the basic idea was to extend JUnit (this in itself was hard since JUnit 3 was not really written for extensibility to distribute test cases to worker machines using JavaSpaces. The idea was to distribute them in order to get the whole suite to run faster, not to run each test a number of times in order to examine failure rates. One of the nice things was that you didn't have to install your tests on the workers since Jini code mobility ensured the bytecodes were downloaded as necessary. However you did need to make sure your tests were Serializable... More at https://computefarm.dev.java.net/samples.html and https://computefarm.dev.java.net/source/browse/computefarm/trunk/samples/src/java/org/tiling/computefarm/samples/junit/ . I think this would be a nice general application of Hadoop, and we'd be able to avoid the restriction on tests having to be Serializable. However, for the immediate problem of reducing test execution time on Hudson, Doug's suggestion of writing a custom ant task sounds like it would be more appropriate.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12393352/HADOOP-1257.patch
          against trunk revision 711654.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to cause Findbugs to fail.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/testReport/
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12393352/HADOOP-1257.patch against trunk revision 711654. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs to fail. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/testReport/ Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/console This message is automatically generated.
          Hide
          Konstantin Boudnik added a comment -

          I'm slightly confused with this issue: is there a need for a special MR tests runner? Or this is all about a general unit test runner which will allow to speed up the testing process on the Hudson's side?

          I believe that it is important to make such a distinguish because it would influence the solution a lot. E.g. for a generic parallel execution we don't need any of MR pieces, proposed by the patch in the attachment (unless I've totally misunderstood its purpose)

          Show
          Konstantin Boudnik added a comment - I'm slightly confused with this issue: is there a need for a special MR tests runner? Or this is all about a general unit test runner which will allow to speed up the testing process on the Hudson's side? I believe that it is important to make such a distinguish because it would influence the solution a lot. E.g. for a generic parallel execution we don't need any of MR pieces, proposed by the patch in the attachment (unless I've totally misunderstood its purpose)
          Hide
          Konstantin Boudnik added a comment -

          Also, it seems like a possibility that parallel tests execution will be included into JUnit 4.7 (see HADOOP-4901 comments on that)

          Show
          Konstantin Boudnik added a comment - Also, it seems like a possibility that parallel tests execution will be included into JUnit 4.7 (see HADOOP-4901 comments on that)

            People

            • Assignee:
              Filip Rogaczewski
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:

                Development