Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
As per HADOOP-4035, the Capacity Scheduler should return no task to a TT if a job's high mem requirements are not met. This doesn't always happen. In the Scheduler's assignTasks() method, if a job's map task does not enough memory to run, the Scheduler looks at reduce tasks, and vice-versa. This can result in a case where a reduce task from another job is returned to the TT (if the high-mem job does not have a reduce task to run, for example), thus starving the high-mem job.