Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7678

Ability to enable logging of container memory stats

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0, 3.0.0
    • Fix Version/s: 3.1.0, 2.10.0, 2.9.1, 3.0.1, 2.8.4
    • Component/s: nodemanager
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      YARN-3424 changed logging of memory stats from ContainersMonitorImpl to INFO to DEBUG.
      We have found these log messages to be useful information in Out-of-Memory situations - they provide detail that helps show the memory profile of the container over time, which can be helpful in determining root cause.

      Here's an example message from YARN-3424:

      2015-03-27 09:32:48,905 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 9215 for container-id container_1427462602546_0002_01_000008: 189.8 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used
      

      Propose to change this to use a separate logger for this message, so that we can enable debug logging for this without enabling all of the other debug logging for ContainersMonitorImpl.

        Attachments

        1. YARN-7678.001.patch
          2 kB
          Jim Brennan
        2. YARN-7678-branch-2.001.patch
          2 kB
          Jim Brennan

          Activity

            People

            • Assignee:
              Jim_Brennan Jim Brennan
              Reporter:
              Jim_Brennan Jim Brennan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: