Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12822

Change "Metrics intern cache overflow" log level from WARN to INFO

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: metrics
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Interns.java outputs "Metrics intern cache over flow" warn log for metrics info/tag when the cache reaches the hard-coded limit and the oldest cache is discarded for the first time. I'm thinking this log level can be changed to info because

      • there is no problem when the oldest cache is removed. if the metrics info/tag is not in the cache, simply create it.
      • we cannot configure the maximum size of the cache, so there is no way to stop the warn log.
      1. HADOOP-12822_2.patch
        2 kB
        Andras Bokor
      2. HADOOP-12822.patch
        1 kB
        Andras Bokor

        Issue Links

          Activity

          Hide
          boky01 Andras Bokor added a comment -

          Akira Ajisaka What do you mean here? I changed two log level. That was you meant here? Please check the patch.

          Show
          boky01 Andras Bokor added a comment - Akira Ajisaka What do you mean here? I changed two log level. That was you meant here? Please check the patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 10m 59s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 7m 2s trunk passed
          +1 compile 5m 52s trunk passed with JDK v1.8.0_77
          +1 compile 6m 43s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 21s trunk passed
          +1 mvnsite 1m 0s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 39s trunk passed
          +1 javadoc 0m 53s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 41s the patch passed
          +1 compile 5m 48s the patch passed with JDK v1.8.0_77
          +1 javac 5m 48s the patch passed
          +1 compile 6m 45s the patch passed with JDK v1.7.0_95
          +1 javac 6m 45s the patch passed
          +1 checkstyle 0m 21s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 1m 57s the patch passed
          +1 javadoc 0m 52s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 5s the patch passed with JDK v1.7.0_95
          +1 unit 7m 10s hadoop-common in the patch passed with JDK v1.8.0_77.
          -1 unit 6m 52s hadoop-common in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 24s Patch does not generate ASF License warnings.
          70m 7s



          Reason Tests
          JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798030/HADOOP-12822.patch
          JIRA Issue HADOOP-12822
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux fc1f1581d22a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1ff27f9
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 10m 59s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 2s trunk passed +1 compile 5m 52s trunk passed with JDK v1.8.0_77 +1 compile 6m 43s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 21s trunk passed +1 mvnsite 1m 0s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 39s trunk passed +1 javadoc 0m 53s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 4s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 41s the patch passed +1 compile 5m 48s the patch passed with JDK v1.8.0_77 +1 javac 5m 48s the patch passed +1 compile 6m 45s the patch passed with JDK v1.7.0_95 +1 javac 6m 45s the patch passed +1 checkstyle 0m 21s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 5s the patch passed with JDK v1.7.0_95 +1 unit 7m 10s hadoop-common in the patch passed with JDK v1.8.0_77. -1 unit 6m 52s hadoop-common in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 70m 7s Reason Tests JDK v1.7.0_95 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798030/HADOOP-12822.patch JIRA Issue HADOOP-12822 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fc1f1581d22a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1ff27f9 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9058/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          I changed two log level. That was you meant here?

          Yes. Thank you for creating the patch. One comment: Now we are gradually migrating from LOG4J to SLF4J. Would you make the code as follows to avoid unnecessary call of size() and e.toString() when the loglevel is WARN or higher?

          public class Interns {
            private static final Logger LOG = LoggerFactory.getLogger(Interns.class);
          
                    LOG.info("Metrics intern cache overflow at {} for {}", size(), e);
          
          Show
          ajisakaa Akira Ajisaka added a comment - I changed two log level. That was you meant here? Yes. Thank you for creating the patch. One comment: Now we are gradually migrating from LOG4J to SLF4J. Would you make the code as follows to avoid unnecessary call of size() and e.toString() when the loglevel is WARN or higher? public class Interns { private static final Logger LOG = LoggerFactory.getLogger(Interns.class); LOG.info( "Metrics intern cache overflow at {} for {}" , size(), e);
          Hide
          boky01 Andras Bokor added a comment -

          Akira Ajisaka Please check my new patch.
          Off topic:
          I am just curious, after migration will log4j remain under SLF4J?

          Show
          boky01 Andras Bokor added a comment - Akira Ajisaka Please check my new patch. Off topic: I am just curious, after migration will log4j remain under SLF4J?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 9m 11s trunk passed
          +1 compile 11m 36s trunk passed with JDK v1.8.0_77
          +1 compile 9m 18s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 22s trunk passed
          +1 mvnsite 1m 8s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          +1 findbugs 1m 51s trunk passed
          +1 javadoc 1m 9s trunk passed with JDK v1.8.0_77
          +1 javadoc 1m 17s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 52s the patch passed
          +1 compile 9m 22s the patch passed with JDK v1.8.0_77
          +1 javac 9m 22s the patch passed
          +1 compile 8m 48s the patch passed with JDK v1.7.0_95
          +1 javac 8m 48s the patch passed
          +1 checkstyle 0m 24s the patch passed
          +1 mvnsite 1m 7s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 9s the patch passed
          +1 javadoc 1m 13s the patch passed with JDK v1.8.0_77
          +1 javadoc 1m 15s the patch passed with JDK v1.7.0_95
          -1 unit 11m 18s hadoop-common in the patch failed with JDK v1.8.0_77.
          +1 unit 10m 18s hadoop-common in the patch passed with JDK v1.7.0_95.
          +1 asflicense 0m 25s Patch does not generate ASF License warnings.
          85m 6s



          Reason Tests
          JDK v1.8.0_77 Failed junit tests hadoop.fs.shell.find.TestIname
            hadoop.fs.shell.find.TestName



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:fbe3e86
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798265/HADOOP-12822_2.patch
          JIRA Issue HADOOP-12822
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux fbcec5d22b00 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 437e9d6
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/testReport/
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 9m 11s trunk passed +1 compile 11m 36s trunk passed with JDK v1.8.0_77 +1 compile 9m 18s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 1m 8s trunk passed +1 mvneclipse 0m 15s trunk passed +1 findbugs 1m 51s trunk passed +1 javadoc 1m 9s trunk passed with JDK v1.8.0_77 +1 javadoc 1m 17s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 52s the patch passed +1 compile 9m 22s the patch passed with JDK v1.8.0_77 +1 javac 9m 22s the patch passed +1 compile 8m 48s the patch passed with JDK v1.7.0_95 +1 javac 8m 48s the patch passed +1 checkstyle 0m 24s the patch passed +1 mvnsite 1m 7s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 9s the patch passed +1 javadoc 1m 13s the patch passed with JDK v1.8.0_77 +1 javadoc 1m 15s the patch passed with JDK v1.7.0_95 -1 unit 11m 18s hadoop-common in the patch failed with JDK v1.8.0_77. +1 unit 10m 18s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 85m 6s Reason Tests JDK v1.8.0_77 Failed junit tests hadoop.fs.shell.find.TestIname   hadoop.fs.shell.find.TestName Subsystem Report/Notes Docker Image:yetus/hadoop:fbe3e86 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12798265/HADOOP-12822_2.patch JIRA Issue HADOOP-12822 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fbcec5d22b00 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 437e9d6 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_77 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_77.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9065/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          LGTM, +1.

          I am just curious, after migration will log4j remain under SLF4J?

          Yes. SLF4J is using LOG4J via slf4j-log4j12.

          Show
          ajisakaa Akira Ajisaka added a comment - LGTM, +1. I am just curious, after migration will log4j remain under SLF4J? Yes. SLF4J is using LOG4J via slf4j-log4j12.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to branch-2.8 and above. Thanks Andras Bokor for the contribution!

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to branch-2.8 and above. Thanks Andras Bokor for the contribution!
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9608 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9608/)
          HADOOP-12822. Change "Metrics intern cache overflow" log level from WARN (aajisaka: rev 27b131e79c5fa99de3ed4fb529d854dd5da55bde)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/Interns.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9608 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9608/ ) HADOOP-12822 . Change "Metrics intern cache overflow" log level from WARN (aajisaka: rev 27b131e79c5fa99de3ed4fb529d854dd5da55bde) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/lib/Interns.java

            People

            • Assignee:
              boky01 Andras Bokor
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development