Details
-
Improvement
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
1.0.3
-
None
-
-
multipleoutputs, multipletestoutputformat, new api, lazyoutputformat
Description
In the new API, and using MultipleOutputs it is possible to segment output into directories by using MultipleOutputs.write(KEYOUT key, VALUEOUT value, String baseOutputPath) in the Reducer to determine the output directory, and by using LazyOutputFormat at the job-level config to suppress normal output [eg use LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class); instead of job.setOutputFormatClass(TextOutputFormat.class);]
This recreates the functionality previously provided in the old API by using MultipleTextOutputFormat (etc)