|
[
Permlink
| « Hide
]
Doug Cutting added a comment - 13/Apr/07 05:53 PM
Bootstrapping could be a little complicated here: do we use an older version of Hadoop to test the new version? Since Hudson runs on a 4-processor box, we might get some mileage by simply getting junit tests to run in parallel. Ant's parallel task is almost appropriate, except there's no easy way I can see to get each junit test to be a sub-task of a parallel task. But perhaps a simple parallel-junit task could be written for this. Test could still be run in separate JVMs, and we could run as many at once as there are CPUs (a feature of the parallel task). A 4x speedup would be very welcome!
> Tom, haven't you done something similar with your https://
> computefarm.dev.java.net/ project? Yes, this was some time ago now, but the basic idea was to extend JUnit (this in itself was hard since JUnit 3 was not really written for extensibility One of the nice things was that you didn't have to install your tests on the workers since Jini code mobility ensured the bytecodes were downloaded as necessary. However you did need to make sure your tests were Serializable... More at https://computefarm.dev.java.net/samples.html I think this would be a nice general application of Hadoop, and we'd be able to avoid the restriction on tests having to be Serializable. However, for the immediate problem of reducing test execution time on Hudson, Doug's suggestion of writing a custom ant task sounds like it would be more appropriate.
Tauhida Parveen made changes - 05/Nov/08 04:35 AM
Tauhida Parveen made changes - 05/Nov/08 05:15 AM
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12393352/HADOOP-1257.patch  against trunk revision 711654. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs to fail. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3537/testReport/ This message is automatically generated.
Nigel Daley made changes - 06/Nov/08 07:07 AM
I'm slightly confused with this issue: is there a need for a special MR tests runner? Or this is all about a general unit test runner which will allow to speed up the testing process on the Hudson's side?
I believe that it is important to make such a distinguish because it would influence the solution a lot. E.g. for a generic parallel execution we don't need any of MR pieces, proposed by the patch in the attachment (unless I've totally misunderstood its purpose) Also, it seems like a possibility that parallel tests execution will be included into JUnit 4.7 (see HADOOP-4901 comments on that)
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
New Feature
Open
Major
a distributed junit test runner
to distribute test cases to worker machines using JavaSpaces. The idea was to distribute them in order to get the whole suite to run faster, not to run each test a number of times in order to examine failure rates.