Hadoop Common
  1. Hadoop Common
  2. HADOOP-9946

NumAllSinks metrics shows lower value than NumActiveSinks

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 2.1.0-beta
    • Fix Version/s: None
    • Component/s: metrics
    • Target Version/s:

      Description

      In my hadoop cluster (trunk), output metrics file is as follows:

      $ less /tmp/nodemanager-metrics.out
      
      1377894554661 metricssystem.MetricsSystem: Context=metricssystem, 
      Hostname=hadoop, NumActiveSources=8, NumAllSources=8, NumActiveSinks=1, 
      NumAllSinks=0, Sink_fileNumOps=0, ...
      

      NumAllSinks should be equal to or greater than NumActiveSinks.

      1. HADOOP-9946
        2 kB
        Akira Ajisaka
      2. HADOOP-9946.02.patch
        2 kB
        Akira Ajisaka
      3. HADOOP-9946.patch
        2 kB
        Akira Ajisaka

        Activity

        Hide
        Akira Ajisaka added a comment -

        Attached a patch to call allSinks.put(String, MetricsSink) when sinks.put(String, MetricsSinkAdapter) is called.

        Show
        Akira Ajisaka added a comment - Attached a patch to call allSinks.put(String, MetricsSink) when sinks.put(String, MetricsSinkAdapter) is called.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12602471/HADOOP-9946.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3077//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3077//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12602471/HADOOP-9946.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/3077//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3077//console This message is automatically generated.
        Hide
        Hadoop QA added a comment -



        +1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 33s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 28s There were no new javac warning messages.
        +1 javadoc 9m 32s There were no new javadoc warning messages.
        +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 3s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 34s mvn install still works.
        +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
        +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 2.0.3) warnings.
        +1 common tests 23m 3s Tests passed in hadoop-common.
            59m 52s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12602471/HADOOP-9946.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / 57d9a97
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6399/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6399/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6399/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 33s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 28s There were no new javac warning messages. +1 javadoc 9m 32s There were no new javadoc warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 3s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 34s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 1m 40s The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 common tests 23m 3s Tests passed in hadoop-common.     59m 52s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12602471/HADOOP-9946.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 57d9a97 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6399/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6399/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6399/console This message was automatically generated.
        Hide
        Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 13m 17s trunk passed
        +1 compile 19m 42s trunk passed with JDK v1.8.0_72
        +1 compile 13m 35s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 36s trunk passed
        +1 mvnsite 1m 50s trunk passed
        +1 mvneclipse 0m 24s trunk passed
        +1 findbugs 2m 44s trunk passed
        +1 javadoc 2m 0s trunk passed with JDK v1.8.0_72
        +1 javadoc 1m 54s trunk passed with JDK v1.7.0_95
        +1 mvninstall 1m 16s the patch passed
        +1 compile 18m 42s the patch passed with JDK v1.8.0_72
        +1 javac 18m 42s the patch passed
        +1 compile 13m 48s the patch passed with JDK v1.7.0_95
        +1 javac 13m 48s the patch passed
        +1 checkstyle 0m 35s the patch passed
        +1 mvnsite 1m 43s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 2m 57s the patch passed
        +1 javadoc 1m 48s the patch passed with JDK v1.8.0_72
        +1 javadoc 1m 39s the patch passed with JDK v1.7.0_95
        -1 unit 13m 20s hadoop-common in the patch failed with JDK v1.8.0_72.
        -1 unit 11m 52s hadoop-common in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 34s Patch does not generate ASF License warnings.
        126m 52s



        Reason Tests
        JDK v1.8.0_72 Failed junit tests hadoop.fs.shell.find.TestName
          hadoop.ipc.TestRPCWaitForProxy
          hadoop.ha.TestZKFailoverController
          hadoop.fs.shell.find.TestPrint
          hadoop.fs.shell.find.TestPrint0
          hadoop.fs.shell.find.TestIname
        JDK v1.7.0_95 Failed junit tests hadoop.fs.shell.find.TestName
          hadoop.fs.shell.find.TestPrint
          hadoop.fs.shell.find.TestPrint0



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788641/HADOOP-9946.02.patch
        JIRA Issue HADOOP-9946
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e8bce162ebc7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 748b6c0
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 13m 17s trunk passed +1 compile 19m 42s trunk passed with JDK v1.8.0_72 +1 compile 13m 35s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 50s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 2m 44s trunk passed +1 javadoc 2m 0s trunk passed with JDK v1.8.0_72 +1 javadoc 1m 54s trunk passed with JDK v1.7.0_95 +1 mvninstall 1m 16s the patch passed +1 compile 18m 42s the patch passed with JDK v1.8.0_72 +1 javac 18m 42s the patch passed +1 compile 13m 48s the patch passed with JDK v1.7.0_95 +1 javac 13m 48s the patch passed +1 checkstyle 0m 35s the patch passed +1 mvnsite 1m 43s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 57s the patch passed +1 javadoc 1m 48s the patch passed with JDK v1.8.0_72 +1 javadoc 1m 39s the patch passed with JDK v1.7.0_95 -1 unit 13m 20s hadoop-common in the patch failed with JDK v1.8.0_72. -1 unit 11m 52s hadoop-common in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 34s Patch does not generate ASF License warnings. 126m 52s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.fs.shell.find.TestName   hadoop.ipc.TestRPCWaitForProxy   hadoop.ha.TestZKFailoverController   hadoop.fs.shell.find.TestPrint   hadoop.fs.shell.find.TestPrint0   hadoop.fs.shell.find.TestIname JDK v1.7.0_95 Failed junit tests hadoop.fs.shell.find.TestName   hadoop.fs.shell.find.TestPrint   hadoop.fs.shell.find.TestPrint0 Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788641/HADOOP-9946.02.patch JIRA Issue HADOOP-9946 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e8bce162ebc7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 748b6c0 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8671/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 34s trunk passed
        +1 compile 5m 44s trunk passed with JDK v1.8.0_72
        +1 compile 6m 51s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 24s trunk passed
        +1 mvnsite 1m 1s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 32s trunk passed
        +1 javadoc 0m 51s trunk passed with JDK v1.8.0_72
        +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 39s the patch passed
        +1 compile 5m 38s the patch passed with JDK v1.8.0_72
        +1 javac 5m 38s the patch passed
        +1 compile 6m 32s the patch passed with JDK v1.7.0_95
        +1 javac 6m 32s the patch passed
        +1 checkstyle 0m 20s the patch passed
        +1 mvnsite 1m 1s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 47s the patch passed
        +1 javadoc 0m 52s the patch passed with JDK v1.8.0_72
        +1 javadoc 1m 0s the patch passed with JDK v1.7.0_95
        +1 unit 6m 39s hadoop-common in the patch passed with JDK v1.8.0_72.
        -1 unit 6m 30s hadoop-common in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        57m 7s



        Reason Tests
        JDK v1.7.0_95 Failed junit tests hadoop.ipc.TestRPCWaitForProxy



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788641/HADOOP-9946.02.patch
        JIRA Issue HADOOP-9946
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux d79d0279845e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 5e7d4d5
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 34s trunk passed +1 compile 5m 44s trunk passed with JDK v1.8.0_72 +1 compile 6m 51s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 24s trunk passed +1 mvnsite 1m 1s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 32s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_72 +1 javadoc 1m 3s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 39s the patch passed +1 compile 5m 38s the patch passed with JDK v1.8.0_72 +1 javac 5m 38s the patch passed +1 compile 6m 32s the patch passed with JDK v1.7.0_95 +1 javac 6m 32s the patch passed +1 checkstyle 0m 20s the patch passed +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 47s the patch passed +1 javadoc 0m 52s the patch passed with JDK v1.8.0_72 +1 javadoc 1m 0s the patch passed with JDK v1.7.0_95 +1 unit 6m 39s hadoop-common in the patch passed with JDK v1.8.0_72. -1 unit 6m 30s hadoop-common in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 57m 7s Reason Tests JDK v1.7.0_95 Failed junit tests hadoop.ipc.TestRPCWaitForProxy Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12788641/HADOOP-9946.02.patch JIRA Issue HADOOP-9946 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux d79d0279845e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 5e7d4d5 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8681/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

          People

          • Assignee:
            Akira Ajisaka
            Reporter:
            Akira Ajisaka
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development