[YARN-7678] Ability to enable logging of container memory stats

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0, 3.0.0
• Fix Version/s: 3.1.0, 2.10.0, 2.9.1, 3.0.1, 2.8.4
• Component/s: nodemanager
• Hadoop Flags: Reviewed
• Assignee: Jim Brennan
• Reporter: Jim Brennan
• Created: 21/Dec/17 16:23
• Updated: 04/Jan/18 17:04
• Resolved: 04/Jan/18 17:04

Description

YARN-3424 changed logging of memory stats from ContainersMonitorImpl to INFO to DEBUG.
We have found these log messages to be useful information in Out-of-Memory situations - they provide detail that helps show the memory profile of the container over time, which can be helpful in determining root cause.

Here's an example message from YARN-3424:

2015-03-27 09:32:48,905 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 9215 for container-id container_1427462602546_0002_01_000008: 189.8 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used

Propose to change this to use a separate logger for this message, so that we can enable debug logging for this without enabling all of the other debug logging for ContainersMonitorImpl.