Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Not A Problem
-
None
-
None
-
None
-
None
Description
I have a MapReduce application with number of reducers equal to the number of machines in the cluster (and with speculative execution turned off). However, Hadoop schedules multiple reduces to run on single machines and leaves other machines idle. This causes contention and seriously slows down the job. Hadoop should employ the simple heuristic of utilizing as many machines as possible when scheduling reduces.