[MAPREDUCE-2469] Task counters should also report the total heap usage of the task

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 0.23.0
• Component/s: task
• Labels: mapreduce
• Hadoop Flags: Reviewed
• Release Note: Task attempt's total heap usage gets recorded and published via counters as COMMITTED_HEAP_BYTES.
• Tags: total-heap, task-counters
• Assignee: Amar Kamat
• Reporter: Amar Kamat
• Created: 03/May/11 04:35
• Updated: 15/Nov/11 00:49
• Resolved: 02/Jun/11 03:16

Description

Currently, the task counters report VSS and RSS usage of the task. The task counter should also report the total heap usage of the task also. The task might be configured with a max heap size of M but the task's total heap usage might only be H, where H < M. In such a case, knowing only M doesn't provide a complete picture of the task's memory usage.

Issue Links

• blocks: MAPREDUCE-2107 Emulate Memory Usage of Tasks in GridMix3
• relates to: MAPREDUCE-2777 Backport MAPREDUCE-220 to Hadoop 20 security branch
• supercedes: MAPREDUCE-2104 Rumen TraceBuilder Does Not Emit CPU/Memory Usage Details in Traces