Description
After one run of a 10 map, 3 reduce sort job, the mapred metrics report:
mapred.tasktracker: hostName=snip, sessionId=, mapTaskSlots=3, maps_running=0, \ reduceTaskSlots=3, reduces_running=0, tasks_completed=15, tasks_failed_ping=0, tasks_failed_timeout=0 mapred.jobtracker: hostName=snip, sessionId=, jobs_completed=1, jobs_submitted=1, \ maps_completed=10, maps_launched=12, reduces_completed=3, reduces_launched=3
After the second (w/ one manually killed reduce)
mapred.tasktracker: hostName=snip, sessionId=, mapTaskSlots=3, maps_running=0, \ reduceTaskSlots=3, reduces_running=0, tasks_completed=32, tasks_failed_ping=0, tasks_failed_timeout=0 mapred.jobtracker: hostName=snip, sessionId=, jobs_completed=2, jobs_submitted=2, \ maps_completed=20, maps_launched=24, reduces_completed=6, reduces_launched=8
The counters report the expected 10/3 map/reduce completions, the second job reporting a single failure.
- The maps_launched and reduces_launched counts are likely recording setup and cleanup tasks as well
- After being recorded among launched tasks, it looks like setup and cleanup are not included among completed tasks
- tasks_completed on the TaskTracker should include only user tasks
- The reduces_launched count makes little sense to me. With three reduces launched per job and one failed, what's the other launched reduce counting?
Attachments
Issue Links
- is part of
-
HADOOP-5514 Add waiting/failed tasks to JobTracker metrics
-
- Closed
-