Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3424

Change logs for ContainerMonitorImpl's resourse monitoring from info to debug

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Today we log the memory usage of process at info level which spams the log with hundreds of log lines

      2015-03-27 09:32:48,905 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 9215 for container-id container_1427462602546_0002_01_000008: 189.8 MB of 1 GB physical memory used; 2.6 GB of 2.1 GB virtual memory used
      

      Proposing changing this to debug level

        Activity

        Hide
        adhoot Anubhav Dhoot added a comment -

        Thanks Tsuyoshi Ozawa for the quick review and commit!

        Show
        adhoot Anubhav Dhoot added a comment - Thanks Tsuyoshi Ozawa for the quick review and commit!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #2100 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2100/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2100 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2100/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #141 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/141/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #141 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/141/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/150/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/150/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2082 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2082/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2082 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2082/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #884 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/884/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #884 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/884/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/150/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #150 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/150/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java hadoop-yarn-project/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7482 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7482/)
        YARN-3424. Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f)

        • hadoop-yarn-project/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7482 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7482/ ) YARN-3424 . Change logs for ContainerMonitorImpl's resourse monitoring from info to debug. Contributed by Anubhav Dhoot. (ozawa: rev c69ba81497ae4da329ddb34ba712a64a7eec479f) hadoop-yarn-project/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Committed this to trunk and branch-2. Thanks Anubhav Dhoot for your contribution!

        Show
        ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Anubhav Dhoot for your contribution!
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        +1 with minor indentation fix on my local. Committing this shortly.

        Show
        ozawa Tsuyoshi Ozawa added a comment - +1 with minor indentation fix on my local. Committing this shortly.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12708505/YARN-3424.001.patch
        against trunk revision 2daa478.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

        Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7186//testReport/
        Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7186//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12708505/YARN-3424.001.patch against trunk revision 2daa478. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/7186//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/7186//console This message is automatically generated.

          People

          • Assignee:
            adhoot Anubhav Dhoot
            Reporter:
            adhoot Anubhav Dhoot
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development