Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12356

Fix computing CPU usage statistics on Windows

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: util
    • Labels:
    • Environment:

      CPU: Intel Xeon
      OS: Windows server

    • Hadoop Flags:
      Reviewed
    • Flags:
      Patch

      Description

      The CPU usage information on Windows is computed incorrectly. The proposed patch fixes the issue, and unifies the the interface with Linux.

      1. 0001-Correct-the-CPU-usage-calcualtion.patch
        3 kB
        Yunqi Zhang
      2. 0001-Correct-the-CPU-usage-calcualtion.patch
        5 kB
        Yunqi Zhang
      3. HADOOP-12356-v3.patch
        5 kB
        Íñigo Goiri
      4. HADOOP-12356-v4.patch
        5 kB
        Íñigo Goiri
      5. HADOOP-12356-v5.patch
        11 kB
        Íñigo Goiri
      6. HADOOP-12356-v6.patch
        11 kB
        Íñigo Goiri
      7. HADOOP-12356-v7.patch
        11 kB
        Íñigo Goiri
      8. HADOOP-12356-v8.patch
        16 kB
        Íñigo Goiri

        Activity

        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-trunk-Commit #9137 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9137/)
        HADOOP-12356. Fix computing CPU usage statistics on Windows. (Inigo (wangda: rev 89d1fd5dac4bccf42d82686e146b02eb60d14736)

        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/MockResourceCalculatorPlugin.java
        • hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/DummyResourceCalculatorPlugin.java
        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeResourceMonitorImpl.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestNodeManagerHardwareUtils.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java
        • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9137 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9137/ ) HADOOP-12356 . Fix computing CPU usage statistics on Windows. (Inigo (wangda: rev 89d1fd5dac4bccf42d82686e146b02eb60d14736) hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/MockResourceCalculatorPlugin.java hadoop-tools/hadoop-gridmix/src/test/java/org/apache/hadoop/mapred/gridmix/DummyResourceCalculatorPlugin.java hadoop-common-project/hadoop-common/CHANGES.txt hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeResourceMonitorImpl.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/util/TestNodeManagerHardwareUtils.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/util/TestSysInfoWindows.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoLinux.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ResourceCalculatorPlugin.java
        Hide
        elgoiri Íñigo Goiri added a comment -

        Thanks Chris Nauroth and Wangda Tan for the review and Yunqi Zhang for the fix!

        Show
        elgoiri Íñigo Goiri added a comment - Thanks Chris Nauroth and Wangda Tan for the review and Yunqi Zhang for the fix!
        Hide
        cnauroth Chris Nauroth added a comment -
        Show
        cnauroth Chris Nauroth added a comment - Thank you, Íñigo Goiri , Yunqi Zhang and Wangda Tan !
        Hide
        leftnoteasy Wangda Tan added a comment -

        Committed to trunk/branch-2/branch-2.8, thanks Íñigo Goiri/Yunqi Zhang and thanks Chris Nauroth for review!

        Show
        leftnoteasy Wangda Tan added a comment - Committed to trunk/branch-2/branch-2.8, thanks Íñigo Goiri / Yunqi Zhang and thanks Chris Nauroth for review!
        Hide
        leftnoteasy Wangda Tan added a comment -

        +1, committing, no failures found in Jenkins result.

        Show
        leftnoteasy Wangda Tan added a comment - +1, committing, no failures found in Jenkins result.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        0 mvndep 1m 5s Maven dependency ordering for branch
        +1 mvninstall 7m 50s trunk passed
        +1 compile 6m 1s trunk passed with JDK v1.8.0_66
        +1 compile 6m 56s trunk passed with JDK v1.7.0_91
        +1 checkstyle 0m 58s trunk passed
        +1 mvnsite 2m 25s trunk passed
        +1 mvneclipse 0m 55s trunk passed
        +1 findbugs 4m 28s trunk passed
        +1 javadoc 1m 50s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 11s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 34s Maven dependency ordering for patch
        +1 mvninstall 2m 48s the patch passed
        +1 compile 6m 2s the patch passed with JDK v1.8.0_66
        +1 javac 6m 2s the patch passed
        +1 compile 6m 49s the patch passed with JDK v1.7.0_91
        +1 javac 6m 49s the patch passed
        +1 checkstyle 0m 57s the patch passed
        +1 mvnsite 2m 24s the patch passed
        +1 mvneclipse 0m 54s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 5m 24s the patch passed
        +1 javadoc 1m 48s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 13s the patch passed with JDK v1.7.0_91
        +1 unit 6m 46s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 1m 56s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        -1 unit 8m 29s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66.
        +1 unit 14m 9s hadoop-gridmix in the patch passed with JDK v1.8.0_66.
        +1 unit 7m 3s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 unit 2m 12s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        -1 unit 9m 0s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_91.
        +1 unit 15m 0s hadoop-gridmix in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        131m 22s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780218/HADOOP-12356-v8.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 88eb3be3d418 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a44ce3f
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-tools/hadoop-gridmix U: .
        Max memory used 77MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 1m 5s Maven dependency ordering for branch +1 mvninstall 7m 50s trunk passed +1 compile 6m 1s trunk passed with JDK v1.8.0_66 +1 compile 6m 56s trunk passed with JDK v1.7.0_91 +1 checkstyle 0m 58s trunk passed +1 mvnsite 2m 25s trunk passed +1 mvneclipse 0m 55s trunk passed +1 findbugs 4m 28s trunk passed +1 javadoc 1m 50s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 11s trunk passed with JDK v1.7.0_91 0 mvndep 0m 34s Maven dependency ordering for patch +1 mvninstall 2m 48s the patch passed +1 compile 6m 2s the patch passed with JDK v1.8.0_66 +1 javac 6m 2s the patch passed +1 compile 6m 49s the patch passed with JDK v1.7.0_91 +1 javac 6m 49s the patch passed +1 checkstyle 0m 57s the patch passed +1 mvnsite 2m 24s the patch passed +1 mvneclipse 0m 54s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 24s the patch passed +1 javadoc 1m 48s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 13s the patch passed with JDK v1.7.0_91 +1 unit 6m 46s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 1m 56s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 8m 29s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66. +1 unit 14m 9s hadoop-gridmix in the patch passed with JDK v1.8.0_66. +1 unit 7m 3s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 2m 12s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. -1 unit 9m 0s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_91. +1 unit 15m 0s hadoop-gridmix in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 131m 22s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780218/HADOOP-12356-v8.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 88eb3be3d418 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a44ce3f Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-tools/hadoop-gridmix U: . Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8438/console This message was automatically generated.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks Chris Nauroth, manually kicked Jenkins build, will commit once Jenkins gets back.

        Show
        leftnoteasy Wangda Tan added a comment - Thanks Chris Nauroth , manually kicked Jenkins build, will commit once Jenkins gets back.
        Hide
        cnauroth Chris Nauroth added a comment -

        Good catch. I didn't realize SysInfo was new in branch-2.8, so no need to preserve interface compatibility with something that didn't exist in any prior release.

        Once again, I am +1 for patch v8.

        Show
        cnauroth Chris Nauroth added a comment - Good catch. I didn't realize SysInfo was new in branch-2.8, so no need to preserve interface compatibility with something that didn't exist in any prior release. Once again, I am +1 for patch v8.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Chris Nauroth/Íñigo Goiri,

        SysInfo is a new interface added to 2.8.0, I think we don't need to mark it to be deprecated.

        Thoughts?

        Show
        leftnoteasy Wangda Tan added a comment - Hi Chris Nauroth / Íñigo Goiri , SysInfo is a new interface added to 2.8.0, I think we don't need to mark it to be deprecated. Thoughts?
        Hide
        cnauroth Chris Nauroth added a comment -

        Would it be OK to add the old method back, mark it as deprecated, and keep the new one?

        That sounds fine to me. Wangda Tan?

        Show
        cnauroth Chris Nauroth added a comment - Would it be OK to add the old method back, mark it as deprecated, and keep the new one? That sounds fine to me. Wangda Tan ?
        Hide
        elgoiri Íñigo Goiri added a comment -

        Would it be OK to add the old method back, mark it as deprecated, and keep
        the new one?

        Show
        elgoiri Íñigo Goiri added a comment - Would it be OK to add the old method back, mark it as deprecated, and keep the new one?
        Hide
        cnauroth Chris Nauroth added a comment -

        I am also +1 for v8.

        I have just one other minor note. SysInfo is annotated Public and Evolving. I prefer the clearer naming in patch v8, but changing the method name is backwards-incompatible. We're within our compatibility policy to do that on a minor release line, so it's fine for 2.8.0, but it cannot be backported to any prior maintenance lines. As long as we're fine with that limitation, I remain +1.

        Wangda Tan, I'd appreciate if you could do the commit, given that this touches a bit of YARN code that you're more familiar with.

        Show
        cnauroth Chris Nauroth added a comment - I am also +1 for v8. I have just one other minor note. SysInfo is annotated Public and Evolving . I prefer the clearer naming in patch v8, but changing the method name is backwards-incompatible. We're within our compatibility policy to do that on a minor release line, so it's fine for 2.8.0, but it cannot be backported to any prior maintenance lines. As long as we're fine with that limitation, I remain +1. Wangda Tan , I'd appreciate if you could do the commit, given that this touches a bit of YARN code that you're more familiar with.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Thanks Íñigo Goiri,

        +1 for latest patch, I think v8 is slightly better than v7, but I don't have strong opinion on that.

        Thoughts? Chris Nauroth.

        Show
        leftnoteasy Wangda Tan added a comment - Thanks Íñigo Goiri , +1 for latest patch, I think v8 is slightly better than v7, but I don't have strong opinion on that. Thoughts? Chris Nauroth .
        Hide
        elgoiri Íñigo Goiri added a comment -

        I don't think the failed unit test is related with the patch.

        Regarding the patch itself, not sure if v8 is better than v7 as it modifies
        many other methods than the initial intent. You guys call it; I don't have
        a strong opinion.

        Show
        elgoiri Íñigo Goiri added a comment - I don't think the failed unit test is related with the patch. Regarding the patch itself, not sure if v8 is better than v7 as it modifies many other methods than the initial intent. You guys call it; I don't have a strong opinion.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
        +1 mvninstall 7m 52s trunk passed
        +1 compile 8m 43s trunk passed with JDK v1.8.0_66
        +1 compile 9m 24s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 2s trunk passed
        +1 mvnsite 2m 36s trunk passed
        +1 mvneclipse 0m 56s trunk passed
        +1 findbugs 4m 47s trunk passed
        +1 javadoc 1m 59s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 21s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 51s the patch passed
        +1 compile 8m 42s the patch passed with JDK v1.8.0_66
        +1 javac 8m 42s the patch passed
        +1 compile 9m 23s the patch passed with JDK v1.7.0_91
        +1 javac 9m 23s the patch passed
        +1 checkstyle 1m 1s the patch passed
        +1 mvnsite 2m 36s the patch passed
        +1 mvneclipse 0m 57s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 5m 24s the patch passed
        +1 javadoc 1m 59s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 19s the patch passed with JDK v1.7.0_91
        -1 unit 8m 1s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        -1 unit 8m 47s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66.
        +1 unit 14m 40s hadoop-gridmix in the patch passed with JDK v1.8.0_66.
        -1 unit 7m 47s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 2m 18s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 9m 15s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 unit 15m 40s hadoop-gridmix in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 25s Patch does not generate ASF License warnings.
        145m 41s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.ha.TestZKFailoverController
          hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService
        JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780218/HADOOP-12356-v8.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux c20e44fd1183 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4e4b3a8
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-gridmix hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 7m 52s trunk passed +1 compile 8m 43s trunk passed with JDK v1.8.0_66 +1 compile 9m 24s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 2s trunk passed +1 mvnsite 2m 36s trunk passed +1 mvneclipse 0m 56s trunk passed +1 findbugs 4m 47s trunk passed +1 javadoc 1m 59s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 21s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 51s the patch passed +1 compile 8m 42s the patch passed with JDK v1.8.0_66 +1 javac 8m 42s the patch passed +1 compile 9m 23s the patch passed with JDK v1.7.0_91 +1 javac 9m 23s the patch passed +1 checkstyle 1m 1s the patch passed +1 mvnsite 2m 36s the patch passed +1 mvneclipse 0m 57s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 5m 24s the patch passed +1 javadoc 1m 59s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 19s the patch passed with JDK v1.7.0_91 -1 unit 8m 1s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 8m 47s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66. +1 unit 14m 40s hadoop-gridmix in the patch passed with JDK v1.8.0_66. -1 unit 7m 47s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 2m 18s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 9m 15s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 unit 15m 40s hadoop-gridmix in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 145m 41s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.yarn.server.nodemanager.containermanager.localizer.TestResourceLocalizationService JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780218/HADOOP-12356-v8.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux c20e44fd1183 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4e4b3a8 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-gridmix hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8330/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Fixing compilation

        Show
        elgoiri Íñigo Goiri added a comment - Fixing compilation
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
        +1 mvninstall 7m 54s trunk passed
        +1 compile 8m 42s trunk passed with JDK v1.8.0_66
        +1 compile 9m 26s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 2s trunk passed
        +1 mvnsite 2m 9s trunk passed
        +1 mvneclipse 0m 42s trunk passed
        +1 findbugs 4m 15s trunk passed
        +1 javadoc 1m 48s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 6s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 32s the patch passed
        -1 compile 8m 2s root in the patch failed with JDK v1.8.0_66.
        -1 javac 8m 2s root in the patch failed with JDK v1.8.0_66.
        -1 compile 8m 46s root in the patch failed with JDK v1.7.0_91.
        -1 javac 8m 46s root in the patch failed with JDK v1.7.0_91.
        +1 checkstyle 1m 2s the patch passed
        +1 mvnsite 2m 9s the patch passed
        +1 mvneclipse 0m 42s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 4m 40s the patch passed
        +1 javadoc 1m 45s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 2s the patch passed with JDK v1.7.0_91
        -1 unit 7m 45s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 45s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        -1 unit 7m 50s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 2m 17s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 9m 15s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        109m 41s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
        JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780214/HADOOP-12356-v8.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 2db2f6f7462c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4e4b3a8
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. +1 mvninstall 7m 54s trunk passed +1 compile 8m 42s trunk passed with JDK v1.8.0_66 +1 compile 9m 26s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 2s trunk passed +1 mvnsite 2m 9s trunk passed +1 mvneclipse 0m 42s trunk passed +1 findbugs 4m 15s trunk passed +1 javadoc 1m 48s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 6s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 32s the patch passed -1 compile 8m 2s root in the patch failed with JDK v1.8.0_66. -1 javac 8m 2s root in the patch failed with JDK v1.8.0_66. -1 compile 8m 46s root in the patch failed with JDK v1.7.0_91. -1 javac 8m 46s root in the patch failed with JDK v1.7.0_91. +1 checkstyle 1m 2s the patch passed +1 mvnsite 2m 9s the patch passed +1 mvneclipse 0m 42s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 40s the patch passed +1 javadoc 1m 45s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 2s the patch passed with JDK v1.7.0_91 -1 unit 7m 45s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 2m 1s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 45s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. -1 unit 7m 50s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 2m 17s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 9m 15s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 109m 41s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780214/HADOOP-12356-v8.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 2db2f6f7462c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4e4b3a8 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8329/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Fixing compilation issue

        Show
        elgoiri Íñigo Goiri added a comment - Fixing compilation issue
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 10s trunk passed
        +1 compile 8m 47s trunk passed with JDK v1.8.0_66
        +1 compile 9m 23s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 3s trunk passed
        +1 mvnsite 2m 9s trunk passed
        +1 mvneclipse 0m 42s trunk passed
        +1 findbugs 4m 14s trunk passed
        +1 javadoc 1m 47s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 2s trunk passed with JDK v1.7.0_91
        -1 mvninstall 0m 22s hadoop-yarn-server-nodemanager in the patch failed.
        -1 compile 3m 32s root in the patch failed with JDK v1.8.0_66.
        -1 javac 3m 32s root in the patch failed with JDK v1.8.0_66.
        -1 compile 3m 48s root in the patch failed with JDK v1.7.0_91.
        -1 javac 3m 48s root in the patch failed with JDK v1.7.0_91.
        +1 checkstyle 1m 1s the patch passed
        -1 mvnsite 0m 25s hadoop-yarn-server-nodemanager in the patch failed.
        +1 mvneclipse 0m 37s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        -1 findbugs 0m 24s hadoop-yarn-server-nodemanager in the patch failed.
        +1 javadoc 1m 42s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 57s the patch passed with JDK v1.7.0_91
        -1 unit 7m 39s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 2m 0s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        -1 unit 0m 25s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66.
        -1 unit 7m 46s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        -1 unit 0m 28s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        81m 51s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics
        JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780203/HADOOP-12356-v8.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5a8dd5202634 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 4e4b3a8
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        findbugs v3.0.0
        mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt
        compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt
        mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 10s trunk passed +1 compile 8m 47s trunk passed with JDK v1.8.0_66 +1 compile 9m 23s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 3s trunk passed +1 mvnsite 2m 9s trunk passed +1 mvneclipse 0m 42s trunk passed +1 findbugs 4m 14s trunk passed +1 javadoc 1m 47s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 2s trunk passed with JDK v1.7.0_91 -1 mvninstall 0m 22s hadoop-yarn-server-nodemanager in the patch failed. -1 compile 3m 32s root in the patch failed with JDK v1.8.0_66. -1 javac 3m 32s root in the patch failed with JDK v1.8.0_66. -1 compile 3m 48s root in the patch failed with JDK v1.7.0_91. -1 javac 3m 48s root in the patch failed with JDK v1.7.0_91. +1 checkstyle 1m 1s the patch passed -1 mvnsite 0m 25s hadoop-yarn-server-nodemanager in the patch failed. +1 mvneclipse 0m 37s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. -1 findbugs 0m 24s hadoop-yarn-server-nodemanager in the patch failed. +1 javadoc 1m 42s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 57s the patch passed with JDK v1.7.0_91 -1 unit 7m 39s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 2m 0s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. -1 unit 0m 25s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.8.0_66. -1 unit 7m 46s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 2m 15s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. -1 unit 0m 28s hadoop-yarn-server-nodemanager in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 81m 51s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics JDK v1.7.0_91 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12780203/HADOOP-12356-v8.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5a8dd5202634 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4e4b3a8 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.8.0_66.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-compile-root-jdk1.7.0_91.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8328/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Changed to getNumVCoresUsed() and getCpuPercentage()

        Show
        elgoiri Íñigo Goiri added a comment - Changed to getNumVCoresUsed() and getCpuPercentage()
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Chris Nauroth,

        Apologize that I missed your last comment, few suggestions regarding names:

        • Is it better to rename ResourceCalculatorPlugin/SysInfo#getCpuUsage to getCpuUsagePercentage?
        • Similarily, is it better to rename ResourceCalculatorPlugin/SysInfo#getVCoresUsage to getNumVCoresUsed?
        Show
        leftnoteasy Wangda Tan added a comment - Hi Chris Nauroth , Apologize that I missed your last comment, few suggestions regarding names: Is it better to rename ResourceCalculatorPlugin/SysInfo# getCpuUsage to getCpuUsagePercentage ? Similarily, is it better to rename ResourceCalculatorPlugin/SysInfo# getVCoresUsage to getNumVCoresUsed ?
        Hide
        cnauroth Chris Nauroth added a comment -

        +1 for patch v7. This looks good to me. Wangda Tan, do you agree?

        Show
        cnauroth Chris Nauroth added a comment - +1 for patch v7. This looks good to me. Wangda Tan , do you agree?
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 8m 1s trunk passed
        +1 compile 7m 59s trunk passed with JDK v1.8.0_66
        +1 compile 8m 40s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 0s trunk passed
        +1 mvnsite 2m 4s trunk passed
        +1 mvneclipse 0m 40s trunk passed
        +1 findbugs 3m 58s trunk passed
        +1 javadoc 1m 37s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 58s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 41s the patch passed
        +1 compile 7m 44s the patch passed with JDK v1.8.0_66
        +1 javac 7m 44s the patch passed
        +1 compile 8m 43s the patch passed with JDK v1.7.0_91
        +1 javac 8m 43s the patch passed
        +1 checkstyle 0m 57s the patch passed
        +1 mvnsite 2m 1s the patch passed
        +1 mvneclipse 0m 40s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 4m 23s the patch passed
        +1 javadoc 1m 38s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 57s the patch passed with JDK v1.7.0_91
        -1 unit 6m 21s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 1m 51s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 31s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        -1 unit 6m 38s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 2m 7s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 8m 58s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        102m 59s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.test.TestTimedOutTestsListener
        JDK v1.7.0_91 Failed junit tests hadoop.test.TestTimedOutTestsListener
          hadoop.ipc.TestIPC



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778886/HADOOP-12356-v7.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 8ba658a96df7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a0de702
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 8m 1s trunk passed +1 compile 7m 59s trunk passed with JDK v1.8.0_66 +1 compile 8m 40s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 0s trunk passed +1 mvnsite 2m 4s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 3m 58s trunk passed +1 javadoc 1m 37s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 58s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 41s the patch passed +1 compile 7m 44s the patch passed with JDK v1.8.0_66 +1 javac 7m 44s the patch passed +1 compile 8m 43s the patch passed with JDK v1.7.0_91 +1 javac 8m 43s the patch passed +1 checkstyle 0m 57s the patch passed +1 mvnsite 2m 1s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 23s the patch passed +1 javadoc 1m 38s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 57s the patch passed with JDK v1.7.0_91 -1 unit 6m 21s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 51s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 31s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. -1 unit 6m 38s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 2m 7s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 8m 58s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 102m 59s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.test.TestTimedOutTestsListener JDK v1.7.0_91 Failed junit tests hadoop.test.TestTimedOutTestsListener   hadoop.ipc.TestIPC Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778886/HADOOP-12356-v7.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8ba658a96df7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a0de702 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8290/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Fixing unavailable values.

        Show
        elgoiri Íñigo Goiri added a comment - Fixing unavailable values.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 54s trunk passed
        +1 compile 8m 39s trunk passed with JDK v1.8.0_66
        +1 compile 9m 25s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 3s trunk passed
        +1 mvnsite 2m 9s trunk passed
        +1 mvneclipse 0m 42s trunk passed
        +1 findbugs 4m 17s trunk passed
        +1 javadoc 1m 44s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 3s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 41s the patch passed
        +1 compile 8m 39s the patch passed with JDK v1.8.0_66
        +1 javac 8m 39s the patch passed
        +1 compile 9m 21s the patch passed with JDK v1.7.0_91
        +1 javac 9m 21s the patch passed
        +1 checkstyle 1m 2s the patch passed
        +1 mvnsite 2m 11s the patch passed
        +1 mvneclipse 0m 41s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 4m 43s the patch passed
        +1 javadoc 1m 43s the patch passed with JDK v1.8.0_66
        +1 javadoc 2m 3s the patch passed with JDK v1.7.0_91
        -1 unit 7m 39s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 44s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        -1 unit 7m 45s hadoop-common in the patch failed with JDK v1.7.0_91.
        -1 unit 2m 13s hadoop-yarn-common in the patch failed with JDK v1.7.0_91.
        +1 unit 9m 13s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        110m 26s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.util.TestSysInfoLinux
          hadoop.util.TestSysInfoWindows
          hadoop.metrics2.impl.TestGangliaMetrics
        JDK v1.7.0_91 Failed junit tests hadoop.util.TestSysInfoLinux
          hadoop.util.TestSysInfoWindows
          hadoop.fs.shell.TestCopyPreserveFlag
          hadoop.yarn.util.TestFSDownload



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778792/HADOOP-12356-v6.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux aefd6fbbd16a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 52ad912
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 54s trunk passed +1 compile 8m 39s trunk passed with JDK v1.8.0_66 +1 compile 9m 25s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 3s trunk passed +1 mvnsite 2m 9s trunk passed +1 mvneclipse 0m 42s trunk passed +1 findbugs 4m 17s trunk passed +1 javadoc 1m 44s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 3s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 41s the patch passed +1 compile 8m 39s the patch passed with JDK v1.8.0_66 +1 javac 8m 39s the patch passed +1 compile 9m 21s the patch passed with JDK v1.7.0_91 +1 javac 9m 21s the patch passed +1 checkstyle 1m 2s the patch passed +1 mvnsite 2m 11s the patch passed +1 mvneclipse 0m 41s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 43s the patch passed +1 javadoc 1m 43s the patch passed with JDK v1.8.0_66 +1 javadoc 2m 3s the patch passed with JDK v1.7.0_91 -1 unit 7m 39s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 44s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. -1 unit 7m 45s hadoop-common in the patch failed with JDK v1.7.0_91. -1 unit 2m 13s hadoop-yarn-common in the patch failed with JDK v1.7.0_91. +1 unit 9m 13s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 110m 26s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.util.TestSysInfoLinux   hadoop.util.TestSysInfoWindows   hadoop.metrics2.impl.TestGangliaMetrics JDK v1.7.0_91 Failed junit tests hadoop.util.TestSysInfoLinux   hadoop.util.TestSysInfoWindows   hadoop.fs.shell.TestCopyPreserveFlag   hadoop.yarn.util.TestFSDownload Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778792/HADOOP-12356-v6.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux aefd6fbbd16a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 52ad912 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8287/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Fixed patch.

        Show
        elgoiri Íñigo Goiri added a comment - Fixed patch.
        Hide
        elgoiri Íñigo Goiri added a comment -

        The unit test should pass now. The only issue is that technically all the asserts in TestSysInfoLinux are reversed: the expected value should be the first parameter and the obtained value the second one. However, for consistency with the others I kept them in the wrong order.

        Show
        elgoiri Íñigo Goiri added a comment - The unit test should pass now. The only issue is that technically all the asserts in TestSysInfoLinux are reversed: the expected value should be the first parameter and the obtained value the second one. However, for consistency with the others I kept them in the wrong order.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        -1 patch 0m 3s HADOOP-12356 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help.



        Subsystem Report/Notes
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778791/HADOOP-12356-v6.patch
        JIRA Issue HADOOP-12356
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8286/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 patch 0m 3s HADOOP-12356 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778791/HADOOP-12356-v6.patch JIRA Issue HADOOP-12356 Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8286/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Fixed getVCoresUsage.

        Show
        elgoiri Íñigo Goiri added a comment - Fixed getVCoresUsage .
        Hide
        elgoiri Íñigo Goiri added a comment -

        I messed up with the 100. I'll fix it on Monday.

        Show
        elgoiri Íñigo Goiri added a comment - I messed up with the 100. I'll fix it on Monday.
        Hide
        cnauroth Chris Nauroth added a comment -

        Íñigo Goiri, would you please look at the last round of test failures? TestCopyPreserveFlag has been a flaky test, so you can skip that one for now. Thanks!

        Show
        cnauroth Chris Nauroth added a comment - Íñigo Goiri , would you please look at the last round of test failures? TestCopyPreserveFlag has been a flaky test, so you can skip that one for now. Thanks!
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
        +1 mvninstall 7m 34s trunk passed
        +1 compile 8m 14s trunk passed with JDK v1.8.0_66
        +1 compile 8m 48s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 0s trunk passed
        +1 mvnsite 2m 3s trunk passed
        +1 mvneclipse 0m 40s trunk passed
        +1 findbugs 4m 2s trunk passed
        +1 javadoc 1m 39s trunk passed with JDK v1.8.0_66
        +1 javadoc 2m 1s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 35s the patch passed
        +1 compile 8m 6s the patch passed with JDK v1.8.0_66
        +1 javac 8m 6s the patch passed
        +1 compile 8m 47s the patch passed with JDK v1.7.0_91
        +1 javac 8m 47s the patch passed
        +1 checkstyle 0m 58s the patch passed
        +1 mvnsite 2m 2s the patch passed
        +1 mvneclipse 0m 40s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 4m 27s the patch passed
        +1 javadoc 1m 37s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 57s the patch passed with JDK v1.7.0_91
        -1 unit 6m 50s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 30s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        -1 unit 6m 58s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 unit 2m 5s hadoop-yarn-common in the patch passed with JDK v1.7.0_91.
        +1 unit 8m 56s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 23s Patch does not generate ASF License warnings.
        104m 18s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.util.TestSysInfoLinux
          hadoop.fs.shell.TestCopyPreserveFlag
          hadoop.util.TestSysInfoWindows
        JDK v1.7.0_91 Failed junit tests hadoop.util.TestSysInfoLinux
          hadoop.util.TestSysInfoWindows



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778591/HADOOP-12356-v5.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b6f55a165f49 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e63388f
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 75MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 7m 34s trunk passed +1 compile 8m 14s trunk passed with JDK v1.8.0_66 +1 compile 8m 48s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 0s trunk passed +1 mvnsite 2m 3s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 4m 2s trunk passed +1 javadoc 1m 39s trunk passed with JDK v1.8.0_66 +1 javadoc 2m 1s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 35s the patch passed +1 compile 8m 6s the patch passed with JDK v1.8.0_66 +1 javac 8m 6s the patch passed +1 compile 8m 47s the patch passed with JDK v1.7.0_91 +1 javac 8m 47s the patch passed +1 checkstyle 0m 58s the patch passed +1 mvnsite 2m 2s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 4m 27s the patch passed +1 javadoc 1m 37s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 57s the patch passed with JDK v1.7.0_91 -1 unit 6m 50s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 59s hadoop-yarn-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 30s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. -1 unit 6m 58s hadoop-common in the patch failed with JDK v1.7.0_91. +1 unit 2m 5s hadoop-yarn-common in the patch passed with JDK v1.7.0_91. +1 unit 8m 56s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 23s Patch does not generate ASF License warnings. 104m 18s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.util.TestSysInfoLinux   hadoop.fs.shell.TestCopyPreserveFlag   hadoop.util.TestSysInfoWindows JDK v1.7.0_91 Failed junit tests hadoop.util.TestSysInfoLinux   hadoop.util.TestSysInfoWindows Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778591/HADOOP-12356-v5.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b6f55a165f49 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e63388f findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 75MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8281/console This message was automatically generated.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Fixed Windows unit test.

        Show
        elgoiri Íñigo Goiri added a comment - Fixed Windows unit test.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Adding getVCoresUsage

        Show
        elgoiri Íñigo Goiri added a comment - Adding getVCoresUsage
        Hide
        leftnoteasy Wangda Tan added a comment -

        Chris Nauroth, make sense to me!

        Show
        leftnoteasy Wangda Tan added a comment - Chris Nauroth , make sense to me!
        Hide
        cnauroth Chris Nauroth added a comment -

        I think a getVCoresUsage method makes sense. That would also make the code self-documenting, because it would make it clear at the call site that the intent is to work in units of vcores. Wangda Tan, what do you think?

        Show
        cnauroth Chris Nauroth added a comment - I think a getVCoresUsage method makes sense. That would also make the code self-documenting, because it would make it clear at the call site that the intent is to work in units of vcores. Wangda Tan , what do you think?
        Hide
        elgoiri Íñigo Goiri added a comment -

        I think Chris Nauroth is talking about the pointless conversion from cores utilized to %CPU and then back to cores utilized. What would you guys like to do? The easiest solution I see is to add getVCoresUsage to SysInfo and then get that value directly in NodeResourceMonitorImpl. I would keep the current getCpuUsage as 0-100%.

        Show
        elgoiri Íñigo Goiri added a comment - I think Chris Nauroth is talking about the pointless conversion from cores utilized to %CPU and then back to cores utilized. What would you guys like to do? The easiest solution I see is to add getVCoresUsage to SysInfo and then get that value directly in NodeResourceMonitorImpl . I would keep the current getCpuUsage as 0-100%.
        Hide
        leftnoteasy Wangda Tan added a comment -

        Chris Nauroth,

        Am I right in thinking that NodeResourceMonitorImpl is the only caller of SysInfo#getCpuUsage (aside from test code)? If so, then SysInfo would be normalizing to [0, 100], only to have NodeResourceMonitorImpl immediately reverse the operation and change it back to core units. Would it be more helpful for SysInfo to return in core units in the first place?

        Agree, and we have getNumCores, and getNumProcessors. So we can normal cpu usage to 0-100% at anytime if we want.

        Show
        leftnoteasy Wangda Tan added a comment - Chris Nauroth , Am I right in thinking that NodeResourceMonitorImpl is the only caller of SysInfo#getCpuUsage (aside from test code)? If so, then SysInfo would be normalizing to [0, 100] , only to have NodeResourceMonitorImpl immediately reverse the operation and change it back to core units. Would it be more helpful for SysInfo to return in core units in the first place? Agree, and we have getNumCores, and getNumProcessors. So we can normal cpu usage to 0-100% at anytime if we want.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 0s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 36s trunk passed
        +1 compile 7m 57s trunk passed with JDK v1.8.0_66
        +1 compile 8m 55s trunk passed with JDK v1.7.0_91
        +1 checkstyle 1m 1s trunk passed
        +1 mvnsite 1m 36s trunk passed
        +1 mvneclipse 0m 27s trunk passed
        +1 findbugs 2m 42s trunk passed
        +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 25s trunk passed with JDK v1.7.0_91
        +1 mvninstall 2m 5s the patch passed
        +1 compile 8m 11s the patch passed with JDK v1.8.0_66
        +1 javac 8m 11s the patch passed
        +1 compile 9m 6s the patch passed with JDK v1.7.0_91
        +1 javac 9m 6s the patch passed
        +1 checkstyle 0m 57s the patch passed
        +1 mvnsite 1m 31s the patch passed
        +1 mvneclipse 0m 27s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 3m 0s the patch passed
        +1 javadoc 1m 12s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 25s the patch passed with JDK v1.7.0_91
        -1 unit 6m 54s hadoop-common in the patch failed with JDK v1.8.0_66.
        +1 unit 8m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66.
        +1 unit 7m 27s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 unit 8m 59s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 21s Patch does not generate ASF License warnings.
        94m 21s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag
          hadoop.fs.TestSymlinkLocalFSFileContext



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778348/HADOOP-12356-v4.patch
        JIRA Issue HADOOP-12356
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bdf8fc9de44b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f5756a2
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/testReport/
        modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: .
        Max memory used 76MB
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 36s trunk passed +1 compile 7m 57s trunk passed with JDK v1.8.0_66 +1 compile 8m 55s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 1s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 2m 42s trunk passed +1 javadoc 1m 13s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 25s trunk passed with JDK v1.7.0_91 +1 mvninstall 2m 5s the patch passed +1 compile 8m 11s the patch passed with JDK v1.8.0_66 +1 javac 8m 11s the patch passed +1 compile 9m 6s the patch passed with JDK v1.7.0_91 +1 javac 9m 6s the patch passed +1 checkstyle 0m 57s the patch passed +1 mvnsite 1m 31s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 3m 0s the patch passed +1 javadoc 1m 12s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 25s the patch passed with JDK v1.7.0_91 -1 unit 6m 54s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 8m 38s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.8.0_66. +1 unit 7m 27s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 8m 59s hadoop-yarn-server-nodemanager in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 94m 21s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag   hadoop.fs.TestSymlinkLocalFSFileContext Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12778348/HADOOP-12356-v4.patch JIRA Issue HADOOP-12356 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bdf8fc9de44b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f5756a2 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: . Max memory used 76MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8267/console This message was automatically generated.
        Hide
        cnauroth Chris Nauroth added a comment -

        I'm fine with the change of NodeResourceMonitorImpl if SysInfoWindows/Linux both normalized usage to 0-100%.

        Am I right in thinking that NodeResourceMonitorImpl is the only caller of SysInfo#getCpuUsage (aside from test code)? If so, then SysInfo would be normalizing to [0, 100], only to have NodeResourceMonitorImpl immediately reverse the operation and change it back to core units. Would it be more helpful for SysInfo to return in core units in the first place?

        Show
        cnauroth Chris Nauroth added a comment - I'm fine with the change of NodeResourceMonitorImpl if SysInfoWindows/Linux both normalized usage to 0-100%. Am I right in thinking that NodeResourceMonitorImpl is the only caller of SysInfo#getCpuUsage (aside from test code)? If so, then SysInfo would be normalizing to [0, 100] , only to have NodeResourceMonitorImpl immediately reverse the operation and change it back to core units. Would it be more helpful for SysInfo to return in core units in the first place?
        Hide
        leftnoteasy Wangda Tan added a comment -

        The "in progress" state has some issues that only assignee can change its state.

        I just resolved and reopened the issue, you can change state yourself now.

        Show
        leftnoteasy Wangda Tan added a comment - The "in progress" state has some issues that only assignee can change its state. I just resolved and reopened the issue, you can change state yourself now.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Not the best comment ever but I added the 0 to 100 range.

        I think Jenkins should run the YARN tests too. I tried to mark it as PA but I cannot.

        Show
        elgoiri Íñigo Goiri added a comment - Not the best comment ever but I added the 0 to 100 range. I think Jenkins should run the YARN tests too. I tried to mark it as PA but I cannot.
        Hide
        elgoiri Íñigo Goiri added a comment -

        Adding comment to SysInfo#getCpuUsage

        Show
        elgoiri Íñigo Goiri added a comment - Adding comment to SysInfo#getCpuUsage
        Hide
        elgoiri Íñigo Goiri added a comment -

        Adding description to SysInfo#getCpuUsage

        Show
        elgoiri Íñigo Goiri added a comment - Adding description to SysInfo#getCpuUsage
        Hide
        leftnoteasy Wangda Tan added a comment -

        Íñigo Goiri

        We have tested in both Windows and Linux and this patch makes both report values from 0 to 100%.

        Sounds good!

        I still think that it would be better to just do the Commons change here in HADOOP and open a bug for the NodeResourceMonitorImpl in YARN. Wangda Tan what do you think?

        I don't have strong opinions for this, if Jenkins can run YARN tests here, I'm fine with putting changes in the single patch.

        Show
        leftnoteasy Wangda Tan added a comment - Íñigo Goiri We have tested in both Windows and Linux and this patch makes both report values from 0 to 100%. Sounds good! I still think that it would be better to just do the Commons change here in HADOOP and open a bug for the NodeResourceMonitorImpl in YARN. Wangda Tan what do you think? I don't have strong opinions for this, if Jenkins can run YARN tests here, I'm fine with putting changes in the single patch.
        Hide
        elgoiri Íñigo Goiri added a comment -

        We have tested in both Windows and Linux and this patch makes both report values from 0 to 100%.
        Agreed on adding the description, it took me a while to see what was wrong.

        I still think that it would be better to just do the Commons change here in HADOOP and open a bug for the NodeResourceMonitorImpl in YARN. Wangda Tan what do you think?

        Show
        elgoiri Íñigo Goiri added a comment - We have tested in both Windows and Linux and this patch makes both report values from 0 to 100%. Agreed on adding the description, it took me a while to see what was wrong. I still think that it would be better to just do the Commons change here in HADOOP and open a bug for the NodeResourceMonitorImpl in YARN. Wangda Tan what do you think?
        Hide
        leftnoteasy Wangda Tan added a comment -
        Show
        leftnoteasy Wangda Tan added a comment - Íñigo Goiri , it seems we're posting comment in parallel , could you take a look at my comment at https://issues.apache.org/jira/browse/HADOOP-12356?focusedCommentId=15062809&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15062809 and share your thoughts?
        Hide
        leftnoteasy Wangda Tan added a comment -

        Hi Íñigo Goiri/Chris Nauroth,

        Looked at patch and discussions:

        Inigo Goiri, my understanding of the resource monitoring code at this layer is that it needs to track utilization as a percentage, not in units of vcores. I'd also be concerned that changing this calculation could cause unusual effects during a rolling upgrade scenario, when some NodeManagers are running the old calculation and some NodeManagers are running the new calculation. I expect Wangda can give us a more definitive answer.

        Since the ResourceUtilization is in 2.8.0 only, I think the rolling upgrade is not an issue.

        I'm wondering if we have different behaviors in SysInfoLinux and SysInfoWindows to get CPU usage:

        • Linux normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated usage after normalization is 50%.
        • Windows is not normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated usage is 400%.

        I think we should make sure the two SysInfo implementation has consistency behavior, maybe we need add the behavior description to SysInfo#getCPUUsage java docs.

        I'm fine with the change of NodeResourceMonitorImpl if SysInfoWindows/Linux both normalized usage to 0-100%.

        Show
        leftnoteasy Wangda Tan added a comment - Hi Íñigo Goiri / Chris Nauroth , Looked at patch and discussions: Inigo Goiri, my understanding of the resource monitoring code at this layer is that it needs to track utilization as a percentage, not in units of vcores. I'd also be concerned that changing this calculation could cause unusual effects during a rolling upgrade scenario, when some NodeManagers are running the old calculation and some NodeManagers are running the new calculation. I expect Wangda can give us a more definitive answer. Since the ResourceUtilization is in 2.8.0 only, I think the rolling upgrade is not an issue. I'm wondering if we have different behaviors in SysInfoLinux and SysInfoWindows to get CPU usage: Linux normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated usage after normalization is 50%. Windows is not normalized to 0-100%: a node with 8 cores, and usages are all 50%, the returned accumulated usage is 400%. I think we should make sure the two SysInfo implementation has consistency behavior, maybe we need add the behavior description to SysInfo#getCPUUsage java docs. I'm fine with the change of NodeResourceMonitorImpl if SysInfoWindows/Linux both normalized usage to 0-100%.
        Hide
        elgoiri Íñigo Goiri added a comment -

        I see your concern now. I actually thing is better to just commit the SysInfoWindows part and for me to open a YARN bug for the NodeResourceMonitorImpl and let Wangda Tan review that one. I'll upload a new patch.

        Show
        elgoiri Íñigo Goiri added a comment - I see your concern now. I actually thing is better to just commit the SysInfoWindows part and for me to open a YARN bug for the NodeResourceMonitorImpl and let Wangda Tan review that one. I'll upload a new patch.
        Hide
        cnauroth Chris Nauroth added a comment -

        I'm not confident enough about the YARN part of this patch, so I'd like to call in help from someone deeper on YARN. Wangda Tan, would you mind taking a look? I'm +1 for the change in SysInfoWindows, but I'm not sure about the change in NodeResourceMonitorImpl.

        Íñigo Goiri, my understanding of the resource monitoring code at this layer is that it needs to track utilization as a percentage, not in units of vcores. I'd also be concerned that changing this calculation could cause unusual effects during a rolling upgrade scenario, when some NodeManagers are running the old calculation and some NodeManagers are running the new calculation. I expect Wangda can give us a more definitive answer.

        Thanks!

        Show
        cnauroth Chris Nauroth added a comment - I'm not confident enough about the YARN part of this patch, so I'd like to call in help from someone deeper on YARN. Wangda Tan , would you mind taking a look? I'm +1 for the change in SysInfoWindows , but I'm not sure about the change in NodeResourceMonitorImpl . Íñigo Goiri , my understanding of the resource monitoring code at this layer is that it needs to track utilization as a percentage, not in units of vcores. I'd also be concerned that changing this calculation could cause unusual effects during a rolling upgrade scenario, when some NodeManagers are running the old calculation and some NodeManagers are running the new calculation. I expect Wangda can give us a more definitive answer. Thanks!
        Hide
        elgoiri Íñigo Goiri added a comment -
        SysInfoLinux.java
          @Override
          public float getCpuUsage() {
            readProcStatFile();
            float overallCpuUsage = cpuTimeTracker.getCpuTrackerUsagePercent();
            if (overallCpuUsage != CpuTimeTracker.UNAVAILABLE) {
              overallCpuUsage = overallCpuUsage / getNumProcessors();
            }
            return overallCpuUsage;
          }
        

        This shows that it will take the summation of all the CPU percentages and divide it by the number of cores to make a number between 0 and 100.

        Show
        elgoiri Íñigo Goiri added a comment - SysInfoLinux.java @Override public float getCpuUsage() { readProcStatFile(); float overallCpuUsage = cpuTimeTracker.getCpuTrackerUsagePercent(); if (overallCpuUsage != CpuTimeTracker.UNAVAILABLE) { overallCpuUsage = overallCpuUsage / getNumProcessors(); } return overallCpuUsage; } This shows that it will take the summation of all the CPU percentages and divide it by the number of cores to make a number between 0 and 100.
        Hide
        elgoiri Íñigo Goiri added a comment -

        SysInfo#getCpuUsage returns a number between 0 and 100%. The ResourceUtilization returned by NodeResourceMonitorImpl should contain the number of used VCores. To return VCores, we need to take the CPU usage (0-100%) from SysInfo#getCpuUsage, multiply it by the total number of cores, and divide by a 100, right?

        Right now in trunk, this is broken for Linux because when the system is fully utilized, it will return 100, while it should return 4 if it has 4 cores.

        Show
        elgoiri Íñigo Goiri added a comment - SysInfo#getCpuUsage returns a number between 0 and 100%. The ResourceUtilization returned by NodeResourceMonitorImpl should contain the number of used VCores. To return VCores, we need to take the CPU usage (0-100%) from SysInfo#getCpuUsage , multiply it by the total number of cores, and divide by a 100, right? Right now in trunk, this is broken for Linux because when the system is fully utilized, it will return 100, while it should return 4 if it has 4 cores.
        Hide
        cnauroth Chris Nauroth added a comment -

        Hello Yunqi Zhang and Íñigo Goiri. The change in SysInfoWindows looks good to me, but it's not clear to me why it was necessary to change NodeResourceMonitorImpl. The existing code there calls into SysInfo#getCpuUsage, so I expect the change in SysInfoWindows#getCpuUsage is sufficient. In fact, it seems like the NodeResourceMonitorImpl change would break Linux, which was already accounting for number of processors in SysInfoLinux#getCpuUsage.

        Show
        cnauroth Chris Nauroth added a comment - Hello Yunqi Zhang and Íñigo Goiri . The change in SysInfoWindows looks good to me, but it's not clear to me why it was necessary to change NodeResourceMonitorImpl . The existing code there calls into SysInfo#getCpuUsage , so I expect the change in SysInfoWindows#getCpuUsage is sufficient. In fact, it seems like the NodeResourceMonitorImpl change would break Linux, which was already accounting for number of processors in SysInfoLinux#getCpuUsage .
        Hide
        elgoiri Íñigo Goiri added a comment -

        Rebased to trunk.

        Show
        elgoiri Íñigo Goiri added a comment - Rebased to trunk.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 10s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
        +1 javac 7m 51s There were no new javac warning messages.
        +1 javadoc 9m 57s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 1m 41s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 27s mvn install still works.
        +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
        +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        +1 common tests 24m 0s Tests passed in hadoop-common.
        -1 yarn tests 7m 36s Tests failed in hadoop-yarn-server-nodemanager.
            74m 48s  



        Reason Tests
        Failed unit tests hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752337/0001-Correct-the-CPU-usage-calcualtion.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / a4d9acc
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/testReport/
        Java 1.7.0_55
        uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 10s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 51s There were no new javac warning messages. +1 javadoc 9m 57s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 41s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 3m 4s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 common tests 24m 0s Tests passed in hadoop-common. -1 yarn tests 7m 36s Tests failed in hadoop-yarn-server-nodemanager.     74m 48s   Reason Tests Failed unit tests hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752337/0001-Correct-the-CPU-usage-calcualtion.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / a4d9acc hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7526/console This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 18m 10s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 8m 54s There were no new javac warning messages.
        +1 javadoc 11m 9s There were no new javadoc warning messages.
        +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings.
        +1 checkstyle 2m 0s There were no new checkstyle issues.
        +1 whitespace 0m 0s The patch has no lines that end in whitespace.
        +1 install 1m 58s mvn install still works.
        +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse.
        +1 findbugs 3m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
        -1 common tests 24m 46s Tests failed in hadoop-common.
        +1 yarn tests 8m 1s Tests passed in hadoop-yarn-server-nodemanager.
            79m 52s  



        Reason Tests
        Failed unit tests hadoop.security.token.delegation.web.TestWebDelegationToken
          hadoop.metrics2.impl.TestMetricsSystemImpl
          hadoop.util.TestSysInfoWindows



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12752312/0001-Correct-the-CPU-usage-calcualtion.patch
        Optional Tests javadoc javac unit findbugs checkstyle
        git revision trunk / eee0d45
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/artifact/patchprocess/testrun_hadoop-common.txt
        hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/testReport/
        Java 1.7.0_55
        uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/console

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 10s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 54s There were no new javac warning messages. +1 javadoc 11m 9s There were no new javadoc warning messages. +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 2m 0s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 58s mvn install still works. +1 eclipse:eclipse 0m 36s The patch built with eclipse:eclipse. +1 findbugs 3m 49s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 common tests 24m 46s Tests failed in hadoop-common. +1 yarn tests 8m 1s Tests passed in hadoop-yarn-server-nodemanager.     79m 52s   Reason Tests Failed unit tests hadoop.security.token.delegation.web.TestWebDelegationToken   hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.util.TestSysInfoWindows Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12752312/0001-Correct-the-CPU-usage-calcualtion.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / eee0d45 hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/artifact/patchprocess/testrun_hadoop-common.txt hadoop-yarn-server-nodemanager test log https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/testReport/ Java 1.7.0_55 uname Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/7524/console This message was automatically generated.
        Hide
        yunqi Yunqi Zhang added a comment -

        The proposed patch fixes the bug in computing the CPU usage on Windows platform.

        Show
        yunqi Yunqi Zhang added a comment - The proposed patch fixes the bug in computing the CPU usage on Windows platform.
        Hide
        yunqi Yunqi Zhang added a comment -

        In hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java (in trunk), the reported CPU usage is incorrect.

        SysInfoWindows.java
        // This is where we calculate all the usage information
        void refreshIfNeeded() {
          ...
          if (lastCumCpuTimeMs != -1) {
            cpuUsage = (cumulativeCpuTimeMs - lastCumCpuTimeMs)
                / (refreshInterval * 1.0f);
          }
          ...
        }
        

        This cpuUsage is the cumulative CPU usage across all logical cores. For example, this number will range in [0.0, 12.0] if a server has 12 cores. Instead of reporting this raw number, we should do the following to be consistent with SysInfoLinux.java. In SysInfoLinux.java, cpuUsage ranges in [0.0, 1200.0], and the return value of getCpuUsage() ranges from [0.0, 100.0].

        SysInfoWindows.java
        // This is where we calculate all the usage information
        void refreshIfNeeded() {
          ...
          if (lastCumCpuTimeMs != -1) {
            cpuUsage = (cumulativeCpuTimeMs - lastCumCpuTimeMs)
                * 100F / refreshInterval;
          }
          ...
        }
        
        @Override
        public float getCpuUsage() {
          refreshIfNeeded();
          return cpuUsage / numProcessors;
        }
        
        Show
        yunqi Yunqi Zhang added a comment - In hadoop-common/src/main/java/org/apache/hadoop/util/SysInfoWindows.java (in trunk), the reported CPU usage is incorrect. SysInfoWindows.java // This is where we calculate all the usage information void refreshIfNeeded() { ... if (lastCumCpuTimeMs != -1) { cpuUsage = (cumulativeCpuTimeMs - lastCumCpuTimeMs) / (refreshInterval * 1.0f); } ... } This cpuUsage is the cumulative CPU usage across all logical cores. For example, this number will range in [0.0, 12.0] if a server has 12 cores. Instead of reporting this raw number, we should do the following to be consistent with SysInfoLinux.java . In SysInfoLinux.java , cpuUsage ranges in [0.0, 1200.0] , and the return value of getCpuUsage() ranges from [0.0, 100.0] . SysInfoWindows.java // This is where we calculate all the usage information void refreshIfNeeded() { ... if (lastCumCpuTimeMs != -1) { cpuUsage = (cumulativeCpuTimeMs - lastCumCpuTimeMs) * 100F / refreshInterval; } ... } @Override public float getCpuUsage() { refreshIfNeeded(); return cpuUsage / numProcessors; }

          People

          • Assignee:
            elgoiri Íñigo Goiri
            Reporter:
            yunqi Yunqi Zhang
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 2h
              2h
              Remaining:
              Remaining Estimate - 2h
              2h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development