[MAPREDUCE-1739] Collecting CPU and memory usage for MapReduce jobs

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Scott Chen
• Reporter: Scott Chen
• Created: 28/Apr/10 20:43
• Updated: 02/May/13 02:29

Description

MAPREDUCE-220 collects CPU and memory usage for each task.
We can aggregate them to get the information per job. Such information can be used for scheduling, profiling or charging the users based on the resource they consumed.

Here are some information that should be useful:
1. Total CPU cycles (# of giga-cycles)
2. Total Memory occupied time (GB-sec)
3. Maximum peak memory on one task (GB)
4. Maximum peak CPU on one task (GHz)

Thoughts?

Issue Links

• depends upon: MAPREDUCE-220 Collecting cpu and memory usage for MapReduce tasks
• relates to: MAPREDUCE-1808 Have a configurable metric reporting CPU/disk usage per user