Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4544

All the log messages about rolling monitoring interval are shown with WARN level

    Details

    • Hadoop Flags:
      Reviewed

      Description

      About yarn.nodemanager.log-aggregation.roll-monitoring-interval-seconds, there are three log messages corresponding to the value set to this parameter, but all of them are shown with the WARN level.

      (a) disabled (default)

      2016-01-05 22:19:29,062 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggreg     ation.AppLogAggregatorImpl: rollingMonitorInterval is set as -1. The log rolling mornitoring interval is disabled. The logs will be aggregated after this application is finished.
      

      (b) enabled

      2016-01-06 00:41:15,808 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: rollingMonitorInterval is set as 7200. The logs will be aggregated every 7200 seconds
      

      (c) enabled but wrong configuration

      2016-01-06 00:39:50,820 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: rollingMonitorIntervall should be more than or equal to 3600 seconds. Using 3600 seconds instead.
      

      I think it is better to output with WARN only in case (c), but it is ok to output with INFO in case (a) and (b).

      1. YARN-4544.patch
        2 kB
        Takashi Ohnishi
      2. YARN-4544.2.patch
        2 kB
        Takashi Ohnishi

        Activity

        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Attached a patch.

        Show
        bwtakacy Takashi Ohnishi added a comment - Attached a patch.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 39s trunk passed
        +1 compile 0m 25s trunk passed with JDK v1.8.0_66
        +1 compile 0m 28s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 11s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 55s trunk passed
        +1 javadoc 0m 17s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 22s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 24s the patch passed
        +1 compile 0m 21s the patch passed with JDK v1.8.0_66
        +1 javac 0m 21s the patch passed
        +1 compile 0m 25s the patch passed with JDK v1.7.0_91
        +1 javac 0m 25s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 25s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 59s the patch passed
        +1 javadoc 0m 15s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91
        +1 unit 8m 27s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 58s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 16s Patch does not generate ASF License warnings.
        33m 10s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780568/YARN-4544.patch
        JIRA Issue YARN-4544
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a3e3cbeec60d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 96d8f1d
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10155/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10155/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 39s trunk passed +1 compile 0m 25s trunk passed with JDK v1.8.0_66 +1 compile 0m 28s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 11s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 55s trunk passed +1 javadoc 0m 17s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 22s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 24s the patch passed +1 compile 0m 21s the patch passed with JDK v1.8.0_66 +1 javac 0m 21s the patch passed +1 compile 0m 25s the patch passed with JDK v1.7.0_91 +1 javac 0m 25s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 25s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 59s the patch passed +1 javadoc 0m 15s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91 +1 unit 8m 27s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 8m 58s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 16s Patch does not generate ASF License warnings. 33m 10s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780568/YARN-4544.patch JIRA Issue YARN-4544 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a3e3cbeec60d 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 96d8f1d Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10155/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10155/console This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Thanks Takashi Ohnishi for reporting this and creating the patch. It looks good to me.
        I found a typo near the change. Would you fix it?

                LOG.info("rollingMonitorInterval is set as "
                    + configuredRollingMonitorInterval + ". "
                    + "The log rolling mornitoring interval is disabled. "
                    + "The logs will be aggregated after this application is finished.");
        

        mornitoring should be monitoring.

        Show
        ajisakaa Akira Ajisaka added a comment - Thanks Takashi Ohnishi for reporting this and creating the patch. It looks good to me. I found a typo near the change. Would you fix it? LOG.info( "rollingMonitorInterval is set as " + configuredRollingMonitorInterval + ". " + "The log rolling mornitoring interval is disabled. " + "The logs will be aggregated after this application is finished." ); mornitoring should be monitoring.
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Sure!

        I will update the patch

        Show
        bwtakacy Takashi Ohnishi added a comment - Sure! I will update the patch
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Attached the updated patch.

        Show
        bwtakacy Takashi Ohnishi added a comment - Attached the updated patch.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        +1 pending Jenkins. Thanks Takashi.

        Show
        ajisakaa Akira Ajisaka added a comment - +1 pending Jenkins. Thanks Takashi.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 28s trunk passed
        +1 compile 0m 25s trunk passed with JDK v1.8.0_66
        +1 compile 0m 27s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 11s trunk passed
        +1 mvnsite 0m 29s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 0m 54s trunk passed
        +1 javadoc 0m 18s trunk passed with JDK v1.8.0_66
        +1 javadoc 0m 22s trunk passed with JDK v1.7.0_91
        +1 mvninstall 0m 23s the patch passed
        +1 compile 0m 23s the patch passed with JDK v1.8.0_66
        +1 javac 0m 23s the patch passed
        +1 compile 0m 25s the patch passed with JDK vdenied
        +1 javac 0m 25s the patch passed
        +1 checkstyle 0m 10s the patch passed
        +1 mvnsite 0m 26s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 0m 59s the patch passed
        +1 javadoc 0m 15s the patch passed with JDK v1.8.0_66
        +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91
        +1 unit 8m 26s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 56s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 17s Patch does not generate ASF License warnings.
        32m 58s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780961/YARN-4544.2.patch
        JIRA Issue YARN-4544
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 23ccad12959a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6702e7d
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10187/testReport/
        modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-YARN-Build/10187/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 28s trunk passed +1 compile 0m 25s trunk passed with JDK v1.8.0_66 +1 compile 0m 27s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 11s trunk passed +1 mvnsite 0m 29s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 0m 54s trunk passed +1 javadoc 0m 18s trunk passed with JDK v1.8.0_66 +1 javadoc 0m 22s trunk passed with JDK v1.7.0_91 +1 mvninstall 0m 23s the patch passed +1 compile 0m 23s the patch passed with JDK v1.8.0_66 +1 javac 0m 23s the patch passed +1 compile 0m 25s the patch passed with JDK vdenied +1 javac 0m 25s the patch passed +1 checkstyle 0m 10s the patch passed +1 mvnsite 0m 26s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 0m 59s the patch passed +1 javadoc 0m 15s the patch passed with JDK v1.8.0_66 +1 javadoc 0m 19s the patch passed with JDK v1.7.0_91 +1 unit 8m 26s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 8m 56s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 17s Patch does not generate ASF License warnings. 32m 58s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780961/YARN-4544.2.patch JIRA Issue YARN-4544 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 23ccad12959a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6702e7d Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-YARN-Build/10187/testReport/ modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-YARN-Build/10187/console This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committing this.

        Show
        ajisakaa Akira Ajisaka added a comment - Committing this.
        Hide
        ajisakaa Akira Ajisaka added a comment - - edited

        Committed this to trunk, branch-2, and branch-2.8. Thanks Takashi Ohnishi for the contribution!

        Show
        ajisakaa Akira Ajisaka added a comment - - edited Committed this to trunk, branch-2, and branch-2.8. Thanks Takashi Ohnishi for the contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9070 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9070/)
        YARN-4544. All the log messages about rolling monitoring interval are (aajisaka: rev 52b77577c44538c26fb9d0144f0b280be3593a38)

        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java
        • hadoop-yarn-project/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9070 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9070/ ) YARN-4544 . All the log messages about rolling monitoring interval are (aajisaka: rev 52b77577c44538c26fb9d0144f0b280be3593a38) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/logaggregation/AppLogAggregatorImpl.java hadoop-yarn-project/CHANGES.txt
        Hide
        bwtakacy Takashi Ohnishi added a comment -

        Thank you, Akira Ajisaka!

        Show
        bwtakacy Takashi Ohnishi added a comment - Thank you, Akira Ajisaka !

          People

          • Assignee:
            bwtakacy Takashi Ohnishi
            Reporter:
            bwtakacy Takashi Ohnishi
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development