[YARN-2141] [Umbrella] Capture container and node resource consumption

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: nodemanager
• Assignee: Unassigned
• Reporter: Carlo Curino
• Created: 10/Jun/14 00:21
• Updated: 17/Mar/15 17:12

Description

Collecting per-container and per-node resource consumption statistics in a fairly granular manner, and making them available to both infrastructure code (e.g., schedulers) and users (e.g., AMs or directly users via webapps), can facilitate several performance work.

Issue Links

• is related to: YARN-2928 YARN Timeline Service v.2: alpha 1
• is related to: YARN-3332 [Umbrella] Unified Resource Statistics Collection per node
• is related to: YARN-1012 Report NM aggregated container resource utilization in heartbeat
• relates to: YARN-3033 [Collector wireup] Implement NM starting the standalone timeline collector daemon

Sub-Tasks

• Metrics for container's actual memory usage: Closed
• Metrics for container's actual CPU usage: Closed
• Generate histograms in ContainerMetrics for actual container resource usage: Resolved