Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The NameNode can periodically log metrics to help debugging when the cluster is not setup with another metrics monitoring scheme.

      1. HDFS-8880.01.patch
        15 kB
        Arpit Agarwal
      2. HDFS-8880.02.patch
        23 kB
        Arpit Agarwal
      3. HDFS-8880.03.patch
        23 kB
        Arpit Agarwal
      4. HDFS-8880.04.patch
        23 kB
        Arpit Agarwal
      5. namenode-metrics.log
        16 kB
        Arpit Agarwal

        Issue Links

          Activity

          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          Patch to log NameNode metrics once every 10 minutes by default to a separate name-metrics.log appender.

          Metrics logging can be turned off by setting dfs.namenode.metrics.logger.period.seconds to a non-positive value.

          The logging filters out TabularData and CompositeData metrics and also truncates string values at 128 bytes to avoid logging excessively.

          Updated patch with tests to follow soon.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited Patch to log NameNode metrics once every 10 minutes by default to a separate name-metrics.log appender. Metrics logging can be turned off by setting dfs.namenode.metrics.logger.period.seconds to a non-positive value. The logging filters out TabularData and CompositeData metrics and also truncates string values at 128 bytes to avoid logging excessively. Updated patch with tests to follow soon.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Sample log file attached.

          Show
          arpitagarwal Arpit Agarwal added a comment - Sample log file attached.
          Hide
          jnp Jitendra Nath Pandey added a comment -

          +1

          Show
          jnp Jitendra Nath Pandey added a comment - +1
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          .02 patch:

          • Added test cases.
          • Replaced Timer with ScheduledThreadPoolExecutor in NameNode.
          Show
          arpitagarwal Arpit Agarwal added a comment - .02 patch: Added test cases. Replaced Timer with ScheduledThreadPoolExecutor in NameNode.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 50s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 1s The patch appears to include 2 new or modified test files.
          +1 javac 7m 38s There were no new javac warning messages.
          +1 javadoc 9m 37s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 45s The applied patch generated 1 new checkstyle issues (total was 6, now 7).
          -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 4m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 22m 5s Tests failed in hadoop-common.
          -1 hdfs tests 174m 41s Tests failed in hadoop-hdfs.
              241m 46s  



          Reason Tests
          Failed unit tests hadoop.ha.TestZKFailoverController
            hadoop.net.TestNetUtils
          Failed build hadoop-hdfs



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12749988/HDFS-8880.02.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 3ae716f
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11974/testReport/
          Java 1.7.0_55
          uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11974/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 50s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 1s The patch appears to include 2 new or modified test files. +1 javac 7m 38s There were no new javac warning messages. +1 javadoc 9m 37s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 45s The applied patch generated 1 new checkstyle issues (total was 6, now 7). -1 whitespace 0m 1s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 4m 20s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 5s Tests failed in hadoop-common. -1 hdfs tests 174m 41s Tests failed in hadoop-hdfs.     241m 46s   Reason Tests Failed unit tests hadoop.ha.TestZKFailoverController   hadoop.net.TestNetUtils Failed build hadoop-hdfs Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12749988/HDFS-8880.02.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 3ae716f checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/11974/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/11974/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/11974/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 17m 7s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 47s There were no new javac warning messages.
          +1 javadoc 9m 41s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 30s The applied patch generated 1 new checkstyle issues (total was 6, now 7).
          -1 whitespace 0m 1s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 4m 24s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 23m 27s Tests failed in hadoop-common.
          -1 hdfs tests 200m 44s Tests failed in hadoop-hdfs.
              267m 31s  



          Reason Tests
          Failed unit tests hadoop.ha.TestZKFailoverController
            hadoop.net.TestNetUtils
            hadoop.hdfs.util.TestByteArrayManager
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.blockmanagement.TestNodeCount
          Timed out tests org.apache.hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12750735/HDFS-8880.03.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 13604bd
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/diffcheckstylehadoop-common.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12006/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12006/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 17m 7s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 47s There were no new javac warning messages. +1 javadoc 9m 41s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 30s The applied patch generated 1 new checkstyle issues (total was 6, now 7). -1 whitespace 0m 1s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 4m 24s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 23m 27s Tests failed in hadoop-common. -1 hdfs tests 200m 44s Tests failed in hadoop-hdfs.     267m 31s   Reason Tests Failed unit tests hadoop.ha.TestZKFailoverController   hadoop.net.TestNetUtils   hadoop.hdfs.util.TestByteArrayManager   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.blockmanagement.TestNodeCount Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12750735/HDFS-8880.03.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 13604bd checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/diffcheckstylehadoop-common.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12006/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12006/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12006/console This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 19m 7s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 2 new or modified test files.
          +1 javac 7m 43s There were no new javac warning messages.
          +1 javadoc 9m 48s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 2m 9s There were no new checkstyle issues.
          -1 whitespace 0m 1s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 4m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 common tests 22m 43s Tests failed in hadoop-common.
          -1 hdfs tests 178m 32s Tests failed in hadoop-hdfs.
              247m 0s  



          Reason Tests
          Failed unit tests hadoop.net.TestNetUtils
            hadoop.ha.TestZKFailoverController
            hadoop.hdfs.TestAppendSnapshotTruncate
          Timed out tests org.apache.hadoop.cli.TestHDFSCLI



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12750840/HDFS-8880.04.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 13604bd
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12008/artifact/patchprocess/whitespace.txt
          hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12008/artifact/patchprocess/testrun_hadoop-common.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12008/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12008/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12008/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 19m 7s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 2 new or modified test files. +1 javac 7m 43s There were no new javac warning messages. +1 javadoc 9m 48s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 9s There were no new checkstyle issues. -1 whitespace 0m 1s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 4m 25s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 22m 43s Tests failed in hadoop-common. -1 hdfs tests 178m 32s Tests failed in hadoop-hdfs.     247m 0s   Reason Tests Failed unit tests hadoop.net.TestNetUtils   hadoop.ha.TestZKFailoverController   hadoop.hdfs.TestAppendSnapshotTruncate Timed out tests org.apache.hadoop.cli.TestHDFSCLI Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12750840/HDFS-8880.04.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 13604bd whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/12008/artifact/patchprocess/whitespace.txt hadoop-common test log https://builds.apache.org/job/PreCommit-HDFS-Build/12008/artifact/patchprocess/testrun_hadoop-common.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12008/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12008/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12008/console This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          The test failures are unrelated and whitespace issues are due to existing context lines in the patch.

          Show
          arpitagarwal Arpit Agarwal added a comment - The test failures are unrelated and whitespace issues are due to existing context lines in the patch.
          Hide
          jnp Jitendra Nath Pandey added a comment -

          +1

          Show
          jnp Jitendra Nath Pandey added a comment - +1
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          Committed to trunk and branch-2.

          Show
          arpitagarwal Arpit Agarwal added a comment - Committed to trunk and branch-2.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8315/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8315/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-common-project/hadoop-common/src/main/conf/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1021 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1021/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1021 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1021/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/291/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #291 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/291/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/conf/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #288 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/288/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #288 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/288/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2237 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2237/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2237 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2237/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java hadoop-common-project/hadoop-common/src/main/conf/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2218 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2218/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2218 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2218/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java hadoop-common-project/hadoop-common/src/main/conf/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/280/)
          HDFS-8880. NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #280 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/280/ ) HDFS-8880 . NameNode metrics logging. (Arpit Agarwal) (arp: rev a88f31ebf3433392419127816f168136de0a9e77) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/resources/log4j.properties hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeMetricsLogger.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/metrics2/util/MBeans.java hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml hadoop-common-project/hadoop-common/src/main/conf/log4j.properties
          Hide
          aw Allen Wittenauer added a comment -

          Isn't this effectively the same as using the file-based metrics2 sink, except specific to HDFS rather than generic for all services?

          Show
          aw Allen Wittenauer added a comment - Isn't this effectively the same as using the file-based metrics2 sink, except specific to HDFS rather than generic for all services?
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          I hadn't seen it before. Thanks for mentioning it. TheFileContext output format could probably be cleaned up to get the same effect as this change.

          Show
          arpitagarwal Arpit Agarwal added a comment - I hadn't seen it before. Thanks for mentioning it. TheFileContext output format could probably be cleaned up to get the same effect as this change.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          I'd be in favour of improving our own general metrics sinks, rather than adding new stuff to every service

          1. it just adds more stuff to maintain, to test, to document and debug
          2. it's not broadly re-usable
          3. it adds 1/thread service, which in test runs could be many more per vm.

          I note that Coda Hale metrics has a stdout streamer for such purposes.

          If we were to do things with metrics, codahale integration would seem a good strategy (though the transitive LGPL dependency on the ganglia reporter is something to be aware of)

          Show
          stevel@apache.org Steve Loughran added a comment - I'd be in favour of improving our own general metrics sinks, rather than adding new stuff to every service it just adds more stuff to maintain, to test, to document and debug it's not broadly re-usable it adds 1/thread service, which in test runs could be many more per vm. I note that Coda Hale metrics has a stdout streamer for such purposes. If we were to do things with metrics, codahale integration would seem a good strategy (though the transitive LGPL dependency on the ganglia reporter is something to be aware of)
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          I'll address #3 by eliminating the extra thread. I am not opposed to a more general solution, pending which this is still useful. I added this to scratch a personal itch as I often missed textual records of NN metrics stored with the service logs for easy grep'ing by metric name or timestamp. There was no intention to add this to every service.

          Coda Hale slf4jreporter looks particularly interesting but IIUC the reporters also use a polling thread and there'd be at least some code added to each service to instantiate reporters. We can file a Jira for a more general solution as there was some community interest from YARN, and perhaps downstream.

          Show
          arpitagarwal Arpit Agarwal added a comment - I'll address #3 by eliminating the extra thread. I am not opposed to a more general solution, pending which this is still useful. I added this to scratch a personal itch as I often missed textual records of NN metrics stored with the service logs for easy grep'ing by metric name or timestamp. There was no intention to add this to every service. Coda Hale slf4jreporter looks particularly interesting but IIUC the reporters also use a polling thread and there'd be at least some code added to each service to instantiate reporters. We can file a Jira for a more general solution as there was some community interest from YARN, and perhaps downstream.

            People

            • Assignee:
              arpitagarwal Arpit Agarwal
              Reporter:
              arpitagarwal Arpit Agarwal
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development