Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3748

Move CS related nodeUpdate log messages to DEBUG

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.1
    • Fix Version/s: 0.23.1
    • Component/s: mrv2
    • Labels:
      None

      Description

      Currently, the RM has nodeUpdate logs per NM per second such as the following:
      2012-01-27 21:51:32,429 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: nodeUpdate: <nodemanager1>:<port1> clusterResources: memory: 57344
      2012-01-27 21:51:32,510 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: nodeUpdate: <nodemanager2>:<port2> clusterResources: memory: 57344
      2012-01-27 21:51:33,094 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: nodeUpdate: <nodemanager1>:<port1> clusterResources: memory: 57344

      Debugging is difficult with huge amount of logs such as this. These logs need to be moved to DEBUG.

      1. MAPREDUCE-3748.patch
        1 kB
        Ramya Sunil
      2. MAPREDUCE-3748.patch
        1 kB
        Ramya Sunil

        Activity

        Hide
        Ramya Sunil added a comment -

        A simple one liner fix.

        Show
        Ramya Sunil added a comment - A simple one liner fix.
        Hide
        Mahadev konar added a comment -

        @Ramya,
        Can you reupload the patch and also grant license to Apache?

        Show
        Mahadev konar added a comment - @Ramya, Can you reupload the patch and also grant license to Apache?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12512507/MAPREDUCE-3748.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1719//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1719//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512507/MAPREDUCE-3748.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1719//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1719//console This message is automatically generated.
        Hide
        Ramya Sunil added a comment -

        Resubmitting the patch.

        Show
        Ramya Sunil added a comment - Resubmitting the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12512529/MAPREDUCE-3748.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1720//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1720//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12512529/MAPREDUCE-3748.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1720//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1720//console This message is automatically generated.
        Hide
        Arun C Murthy added a comment -

        I just committed this after a minor change to add a isDebugEnabled check. Thanks Ramya!

        Show
        Arun C Murthy added a comment - I just committed this after a minor change to add a isDebugEnabled check. Thanks Ramya!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #1621 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1621/)
        MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #1621 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/1621/ ) MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #1693 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1693/)
        MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #1693 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1693/ ) MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Commit #441 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/441/)
        Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Commit #441 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Commit/441/ ) Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-0.23-Commit #450 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/450/)
        Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-0.23-Commit #450 (See https://builds.apache.org/job/Hadoop-Common-0.23-Commit/450/ ) Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #1638 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1638/)
        MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #1638 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/1638/ ) MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Commit #465 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/465/)
        Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Commit #465 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Commit/465/ ) Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #942 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/942/)
        MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #942 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/942/ ) MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #155 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/155/)
        Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #155 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/155/ ) Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-0.23-Build #177 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/177/)
        Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-0.23-Build #177 (See https://builds.apache.org/job/Hadoop-Mapreduce-0.23-Build/177/ ) Merge -c 1238157 from trunk to branch-0.23 to fix MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238159 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #975 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/975/)
        MAPREDUCE-3748. Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil.

        acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #975 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/975/ ) MAPREDUCE-3748 . Changed a log in CapacityScheduler.nodeUpdate to debug. Contributed by Ramya Sunil. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1238157 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java

          People

          • Assignee:
            Ramya Sunil
            Reporter:
            Ramya Sunil
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development