Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
hadoopStreaming was originally built and tested separately.
It makes sense to keep building the hadoopStreaming code as a separate jar file:
this is all client code that can be uploaded in the Job jar.
this allows quick turnaround on production clusters
(hadoopStreaming code updates do not require to bring down the MapReduce system.
If necessary users can use their own modified versions of the hadoopStreaming jar but still run on the production cluster.)
On the other hand it makes sense to build this code nightly.
Many recent changes broke either compilation or correctness.
All the problems would have been caught if the hadoopStreaming compilation and tests were run nighlty.
Conclusion:
the updated top-level build.xml adds the following dependencies:
1. Target compile calls target compile in contrib/streaming/build.xml
2. Target test calls target test in contrib/streaming/build.xml