Details
-
Bug
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
None
-
None
-
None
Description
Currently, JobControlCompiler makes an estimation for the number of reducers required (by default based on input size into mappers) regardless of whether there is a reduce phase in the job. This is unnecessary, especially when running more complex custom reducer estimators.
Change to only estimate reducers when necessary.