Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8887

Use a Maven plugin to build the native code using CMake

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.3-alpha
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently, we build the native code using ant-build invocations. Although this works, it has some limitations:

      • compiler warning messages are hidden, which can cause people to check in code with warnings unintentionally
      • there is no framework for running native unit tests; instead, we use ad-hoc constructs involving shell scripts
      • the antrun code is very platform specific
      • there is no way to run a specific native unit test
      • it's more or less impossible for scripts like test-patch.sh to separate a native test failing from the build itself failing (no files are created) or to enumerate which native tests failed.

      Using a native Maven plugin would overcome these limitations.

      1. HADOOP-8887.001.patch
        50 kB
        Colin P. McCabe
      2. HADOOP-8887.002.patch
        51 kB
        Colin P. McCabe
      3. HADOOP-8887.003.patch
        54 kB
        Colin P. McCabe
      4. HADOOP-8887.004.patch
        54 kB
        Colin P. McCabe
      5. HADOOP-8887.005.patch
        50 kB
        Colin P. McCabe
      6. HADOOP-8887.006.patch
        50 kB
        Colin P. McCabe
      7. HADOOP-8887.008.patch
        54 kB
        Colin P. McCabe
      8. HADOOP-8887.011.patch
        18 kB
        Colin P. McCabe
      9. HADOOP-8887.012.patch
        43 kB
        Colin P. McCabe
      10. HADOOP-8887.013.patch
        43 kB
        Colin P. McCabe
      11. HADOOP-8887.014.patch
        43 kB
        Colin P. McCabe

        Issue Links

          Activity

          Hide
          cmccabe Colin P. McCabe added a comment -

          Yes, let's take care of the whitespace / etc. stuff on HADOOP-12712. There are also some flaky native unit tests, which I also think should be fixed in a follow-on.

          Show
          cmccabe Colin P. McCabe added a comment - Yes, let's take care of the whitespace / etc. stuff on HADOOP-12712 . There are also some flaky native unit tests, which I also think should be fixed in a follow-on.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9113 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9113/)
          HADOOP-8887. Use a Maven plugin to build the native code using CMake (cmccabe: rev b1ed28fa77cb2fab80c54f9dfeb5d8b7139eca34)

          • hadoop-common-project/hadoop-common/pom.xml
          • hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • BUILDING.txt
          • hadoop-common-project/hadoop-common/CHANGES.txt
          • hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/CompileMojo.java
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml
          • hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java
          • hadoop-tools/hadoop-pipes/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9113 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9113/ ) HADOOP-8887 . Use a Maven plugin to build the native code using CMake (cmccabe: rev b1ed28fa77cb2fab80c54f9dfeb5d8b7139eca34) hadoop-common-project/hadoop-common/pom.xml hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml BUILDING.txt hadoop-common-project/hadoop-common/CHANGES.txt hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/CompileMojo.java hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/pom.xml hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/util/Exec.java hadoop-tools/hadoop-pipes/pom.xml
          Hide
          andrew.wang Andrew Wang added a comment -

          There's a bunch of checkstyle stuff, and findbugs looks like it's broken because of some kafka stuff.

          CC warnings, I'm guessing you're handling at HADOOP-12712 which I'll review.

          Overall it seems okay, it's just there's a lot more "-1" than I'm used to in a precommit run, and it's not just flaky unit tests. The whitespace/checkstyle stuff is onerous, but one more precommit run will fix it.

          If you want to take care of all this in HADOOP-12712, I'm okay with that. Nothing functionally wrong in what was committed, so okay to do it in a follow-on.

          Show
          andrew.wang Andrew Wang added a comment - There's a bunch of checkstyle stuff, and findbugs looks like it's broken because of some kafka stuff. CC warnings, I'm guessing you're handling at HADOOP-12712 which I'll review. Overall it seems okay, it's just there's a lot more "-1" than I'm used to in a precommit run, and it's not just flaky unit tests. The whitespace/checkstyle stuff is onerous, but one more precommit run will fix it. If you want to take care of all this in HADOOP-12712 , I'm okay with that. Nothing functionally wrong in what was committed, so okay to do it in a follow-on.
          Hide
          cmccabe Colin P. McCabe added a comment -

          So, the two issues from the plugin I saw were this deprecation warning:

          [WARNING] /testptch/hadoop/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java:[207,30] [deprecation] getExecutionProperties() in MavenSession has been deprecated
          [WARNING] /testptch/hadoop/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java:[225,29] [deprecation] getExecutionProperties() in MavenSession has been deprecated
          

          and a complaint about the use of a hard tab in TestMojo.java. I opened HADOOP-12712 to fix these warnings.

          Aside from that, we have the usual Jenkins weirdness on one of the test runs.

          java.lang.RuntimeException: Error while running command to get file permissions : ExitCodeException exitCode=127: /bin/ls: error while loading shared libraries: libdl.so.2: failed to map segment from shared object: Permission denied
          

          libdl.so.2 is a Linux system library-- nothing to do with anything in this patch. Best guess is that someone ran "apt-get update" concurrently with Jenkins and hosed this particular unit test.

          Does that cover all the issues?

          Show
          cmccabe Colin P. McCabe added a comment - So, the two issues from the plugin I saw were this deprecation warning: [WARNING] /testptch/hadoop/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java:[207,30] [deprecation] getExecutionProperties() in MavenSession has been deprecated [WARNING] /testptch/hadoop/hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/TestMojo.java:[225,29] [deprecation] getExecutionProperties() in MavenSession has been deprecated and a complaint about the use of a hard tab in TestMojo.java . I opened HADOOP-12712 to fix these warnings. Aside from that, we have the usual Jenkins weirdness on one of the test runs. java.lang.RuntimeException: Error while running command to get file permissions : ExitCodeException exitCode=127: /bin/ls: error while loading shared libraries: libdl.so.2: failed to map segment from shared object: Permission denied libdl.so.2 is a Linux system library-- nothing to do with anything in this patch. Best guess is that someone ran "apt-get update" concurrently with Jenkins and hosed this particular unit test. Does that cover all the issues?
          Hide
          andrew.wang Andrew Wang added a comment -

          Hey Colin, I think commit was a little premature, since precommit flagged a bunch of issues. Could you take a look?

          Show
          andrew.wang Andrew Wang added a comment - Hey Colin, I think commit was a little premature, since precommit flagged a bunch of issues. Could you take a look?
          Hide
          cmccabe Colin P. McCabe added a comment -

          Committed to 2.9, thanks.

          Show
          cmccabe Colin P. McCabe added a comment - Committed to 2.9, thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 1m 54s Maven dependency ordering for branch
          +1 mvninstall 8m 2s trunk passed
          +1 compile 8m 59s trunk passed with JDK v1.8.0_66
          +1 compile 9m 26s trunk passed with JDK v1.7.0_91
          +1 checkstyle 1m 4s trunk passed
          +1 mvnsite 10m 18s trunk passed
          +1 mvneclipse 0m 54s trunk passed
          -1 findbugs 21m 57s root in trunk failed.
          +1 javadoc 7m 24s trunk passed with JDK v1.8.0_66
          +1 javadoc 11m 5s trunk passed with JDK v1.7.0_91
          0 mvndep 1m 16s Maven dependency ordering for patch
          +1 mvninstall 8m 56s the patch passed
          +1 compile 8m 25s the patch passed with JDK v1.8.0_66
          -1 cc 18m 38s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 13, now 13).
          +1 cc 8m 25s the patch passed
          -1 javac 18m 38s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 2 new issues (was 730, now 732).
          +1 javac 8m 25s the patch passed
          +1 compile 8m 38s the patch passed with JDK v1.7.0_91
          -1 cc 27m 17s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 3 new issues (was 23, now 23).
          +1 cc 8m 38s the patch passed
          -1 javac 27m 17s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 724, now 726).
          +1 javac 8m 38s the patch passed
          -1 checkstyle 1m 15s Patch generated 25 new checkstyle issues in root (total was 3, now 28).
          +1 mvnsite 11m 21s the patch passed
          +1 mvneclipse 0m 46s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) with tabs.
          +1 xml 0m 3s The patch has no ill-formed XML file.
          -1 findbugs 22m 41s root in the patch failed.
          -1 javadoc 0m 17s root in the patch failed with JDK v1.8.0_66.
          +1 javadoc 9m 58s the patch passed with JDK v1.7.0_91
          -1 unit 4m 37s root in the patch failed with JDK v1.8.0_66.
          -1 unit 88m 47s root in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 27s Patch does not generate ASF License warnings.
          249m 49s



          Reason Tests
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSInputStream
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
            hadoop.hdfs.server.namenode.TestStartup



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782154/HADOOP-8887.014.patch
          JIRA Issue HADOOP-8887
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle cc
          uname Linux df59419dc9dc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fb64e60
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/branch-findbugs-root.txt
          cc root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-cc-root-jdk1.8.0_66.txt
          javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt
          cc root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-cc-root-jdk1.7.0_91.txt
          javac root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_91.txt
          checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/whitespace-tabs.txt
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-findbugs-root.txt
          javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-javadoc-root-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-unit-root-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-unit-root-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-unit-root-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-maven-plugins hadoop-tools/hadoop-pipes U: .
          Max memory used 120MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 1m 54s Maven dependency ordering for branch +1 mvninstall 8m 2s trunk passed +1 compile 8m 59s trunk passed with JDK v1.8.0_66 +1 compile 9m 26s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 4s trunk passed +1 mvnsite 10m 18s trunk passed +1 mvneclipse 0m 54s trunk passed -1 findbugs 21m 57s root in trunk failed. +1 javadoc 7m 24s trunk passed with JDK v1.8.0_66 +1 javadoc 11m 5s trunk passed with JDK v1.7.0_91 0 mvndep 1m 16s Maven dependency ordering for patch +1 mvninstall 8m 56s the patch passed +1 compile 8m 25s the patch passed with JDK v1.8.0_66 -1 cc 18m 38s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new issues (was 13, now 13). +1 cc 8m 25s the patch passed -1 javac 18m 38s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 2 new issues (was 730, now 732). +1 javac 8m 25s the patch passed +1 compile 8m 38s the patch passed with JDK v1.7.0_91 -1 cc 27m 17s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 3 new issues (was 23, now 23). +1 cc 8m 38s the patch passed -1 javac 27m 17s root-jdk1.7.0_91 with JDK v1.7.0_91 generated 2 new issues (was 724, now 726). +1 javac 8m 38s the patch passed -1 checkstyle 1m 15s Patch generated 25 new checkstyle issues in root (total was 3, now 28). +1 mvnsite 11m 21s the patch passed +1 mvneclipse 0m 46s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) with tabs. +1 xml 0m 3s The patch has no ill-formed XML file. -1 findbugs 22m 41s root in the patch failed. -1 javadoc 0m 17s root in the patch failed with JDK v1.8.0_66. +1 javadoc 9m 58s the patch passed with JDK v1.7.0_91 -1 unit 4m 37s root in the patch failed with JDK v1.8.0_66. -1 unit 88m 47s root in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 27s Patch does not generate ASF License warnings. 249m 49s Reason Tests JDK v1.7.0_91 Failed junit tests hadoop.hdfs.TestDFSInputStream   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark   hadoop.hdfs.server.namenode.TestStartup Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12782154/HADOOP-8887.014.patch JIRA Issue HADOOP-8887 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit xml findbugs checkstyle cc uname Linux df59419dc9dc 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fb64e60 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/branch-findbugs-root.txt cc root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-cc-root-jdk1.8.0_66.txt javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt cc root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-cc-root-jdk1.7.0_91.txt javac root-jdk1.7.0_91: https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_91.txt checkstyle https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/whitespace-tabs.txt findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-findbugs-root.txt javadoc https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-javadoc-root-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-unit-root-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-unit-root-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/artifact/patchprocess/patch-unit-root-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager . hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-maven-plugins hadoop-tools/hadoop-pipes U: . Max memory used 120MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8404/console This message was automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Rather than hardcoding 4, we could use this Java API to get the # of processors: https://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#availableProcessors()

          Cool. I didn't know this API existed... will add.

          Nit: validateSourceParams, switch the order of arguments? Left to right I'd normally read input then output.

          OK.

          Would be good to update BUILDING.txt with your "-DallNative" and "-Dtest_bulk_crc32" invocations as documentation

          Added

          Show
          cmccabe Colin P. McCabe added a comment - Rather than hardcoding 4, we could use this Java API to get the # of processors: https://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#availableProcessors( ) Cool. I didn't know this API existed... will add. Nit: validateSourceParams, switch the order of arguments? Left to right I'd normally read input then output. OK. Would be good to update BUILDING.txt with your "-DallNative" and "-Dtest_bulk_crc32" invocations as documentation Added
          Hide
          andrew.wang Andrew Wang added a comment -

          Nice work here Colin! Just a few review comments:

          Otherwise looks great. I ran the native tests, some Java tests that use the native libs, and did a sanity check on a dist tarball too (libs were in the right place).

          +1 pending this and a Jenkins run.

          Show
          andrew.wang Andrew Wang added a comment - Nice work here Colin! Just a few review comments: Rather than hardcoding 4, we could use this Java API to get the # of processors: https://docs.oracle.com/javase/7/docs/api/java/lang/Runtime.html#availableProcessors( ) Nit: validateSourceParams, switch the order of arguments? Left to right I'd normally read input then output. Would be good to update BUILDING.txt with your "-DallNative" and "-Dtest_bulk_crc32" invocations as documentation Otherwise looks great. I ran the native tests, some Java tests that use the native libs, and did a sanity check on a dist tarball too (libs were in the right place). +1 pending this and a Jenkins run.
          Hide
          cmccabe Colin P. McCabe added a comment - - edited

          uploaded v13: One small revision to take into account the fact that hadoop-hdfs-native-client wants its cmake directory in a non-standard location

          Show
          cmccabe Colin P. McCabe added a comment - - edited uploaded v13: One small revision to take into account the fact that hadoop-hdfs-native-client wants its cmake directory in a non-standard location
          Hide
          drankye Kai Zheng added a comment -

          This sounds so good to have! I felt the same pain and necessity during working on ISA-L integration, having been wondering if it's possible to unify the native building process for both Linux and Windows, provide much easier setup for adding new unit tests until finding this. Would very happily lend a hand in this aspect if necessary. Thanks!

          Show
          drankye Kai Zheng added a comment - This sounds so good to have! I felt the same pain and necessity during working on ISA-L integration, having been wondering if it's possible to unify the native building process for both Linux and Windows, provide much easier setup for adding new unit tests until finding this. Would very happily lend a hand in this aspect if necessary. Thanks!
          Hide
          cmccabe Colin P. McCabe added a comment -

          I rebased this on current trunk and removed some of the hacks that were needed for jdk6. This now integrates with the code in hadoop-maven-plugins.

          I think this JIRA is important because the current method of calling cmake via bash will not work for Windows. Once this plugin is established, we will be able to add Windows support fairly easily in a follow-on JIRA, and no longer have to maintain a separate native build process for Windows.

          The plugin also makes running native unit tests straightforward and natural. We no longer have to duplicate the SKIPTESTS handling and the cmake 2.6 hack everywhere, but can have them in just one place-- the cmake plugin itself.

          With the cmakebuilder maven plugin, you can now run all native unit tests via:

          mvn test -Pnative -Dtest=allNative
          

          or a specific native unit test via:

          mvn test -Pnative -Dtest=test_bulk_crc32
          

          (or whatever other native unit test name you have, clearly) The stdout and stderr show up in .stdout and .stderr files, respectively. Test timeouts are enforced and we track the exit codes.

          Show
          cmccabe Colin P. McCabe added a comment - I rebased this on current trunk and removed some of the hacks that were needed for jdk6. This now integrates with the code in hadoop-maven-plugins. I think this JIRA is important because the current method of calling cmake via bash will not work for Windows. Once this plugin is established, we will be able to add Windows support fairly easily in a follow-on JIRA, and no longer have to maintain a separate native build process for Windows. The plugin also makes running native unit tests straightforward and natural. We no longer have to duplicate the SKIPTESTS handling and the cmake 2.6 hack everywhere, but can have them in just one place-- the cmake plugin itself. With the cmakebuilder maven plugin, you can now run all native unit tests via: mvn test -Pnative -Dtest=allNative or a specific native unit test via: mvn test -Pnative -Dtest=test_bulk_crc32 (or whatever other native unit test name you have, clearly) The stdout and stderr show up in .stdout and .stderr files, respectively. Test timeouts are enforced and we track the exit codes.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Please, -1 on automatic downloading of cmake. The build takes long enough as-is, and cmake takes forever to download and install. Plus we would have to somehow find the correct version for each different Linux distribution / BSD / Mac OS release, which puts this firmly in Not Happening territory. And building cmake from source takes a really long time, maybe almost as much as Hadoop itself! I don't remember exactly how long it took last time I did it.

          I think we could move off of supporting cmake 2.6 in branch-2 today if we wanted to. We only supported it earlier because people wanted to run on Red Hat 5, but that is no longer a concern. We could bump the minimum version up to 2.8 and get rid of a bunch of quirks code.

          Show
          cmccabe Colin P. McCabe added a comment - Please, -1 on automatic downloading of cmake. The build takes long enough as-is, and cmake takes forever to download and install. Plus we would have to somehow find the correct version for each different Linux distribution / BSD / Mac OS release, which puts this firmly in Not Happening territory. And building cmake from source takes a really long time, maybe almost as much as Hadoop itself! I don't remember exactly how long it took last time I did it. I think we could move off of supporting cmake 2.6 in branch-2 today if we wanted to. We only supported it earlier because people wanted to run on Red Hat 5, but that is no longer a concern. We could bump the minimum version up to 2.8 and get rid of a bunch of quirks code.
          Hide
          wheat9 Haohui Mai added a comment -

          Either approach looks reasonable to me. The only benefit of the cmake-maven-project plugin is that it downloads cmake 2.8 automatically thus it is possible to remove a bunch of quirks for cmake 2.6 we have today.

          Show
          wheat9 Haohui Mai added a comment - Either approach looks reasonable to me. The only benefit of the cmake-maven-project plugin is that it downloads cmake 2.8 automatically thus it is possible to remove a bunch of quirks for cmake 2.6 we have today.
          Hide
          cmccabe Colin P. McCabe added a comment -

          I did try to work with cmake-maven-project in the past. I wasn't able to get it working. The biggest issues were the lack of support for JDK6 (which would no longer be an issue, of course), and the fact that it was mainly tested and developed on Windows. I wasn't able to get it to work on UNIX at all, after spending some time with it a few years ago. It looks like the project has not been very active (last commit was more than a year ago), so I don't think there is a big benefit over just using cmake-maven-ng. cmake-maven-ng also supported a mode which allowed all tests to be run, or just a specific native test, something that a large project like Hadoop really needs. Now that we are bundling other maven plugins in the Hadoop repo, we should probably just bundle the cmake-maven-ng plugin there.

          Show
          cmccabe Colin P. McCabe added a comment - I did try to work with cmake-maven-project in the past. I wasn't able to get it working. The biggest issues were the lack of support for JDK6 (which would no longer be an issue, of course), and the fact that it was mainly tested and developed on Windows. I wasn't able to get it to work on UNIX at all, after spending some time with it a few years ago. It looks like the project has not been very active (last commit was more than a year ago), so I don't think there is a big benefit over just using cmake-maven-ng. cmake-maven-ng also supported a mode which allowed all tests to be run, or just a specific native test, something that a large project like Hadoop really needs. Now that we are bundling other maven plugins in the Hadoop repo, we should probably just bundle the cmake-maven-ng plugin there.
          Hide
          wheat9 Haohui Mai added a comment -

          I just came across https://code.google.com/p/cmake-maven-project/ today.

          Colin P. McCabe, do you have any experience on the cmake-maven-project? Do you think it makes sense to adopt the plugin in the current POMs?

          Show
          wheat9 Haohui Mai added a comment - I just came across https://code.google.com/p/cmake-maven-project/ today. Colin P. McCabe , do you have any experience on the cmake-maven-project? Do you think it makes sense to adopt the plugin in the current POMs?
          Hide
          cmccabe Colin P. McCabe added a comment -

          after some consultation with the Maven experts, it seems that including a Maven plugin in the same repository you use it in is not supported.

          I think the way forward here is to maintain cmake-maven-ng as its own Apache project.
          In the meantime, I have put cmake-maven-ng on Apache Extras under http://code.google.com/a/apache-extras.org/p/cmake-maven-ng/. Maven can make use of this repository in the meantime.

          The following patch implements this.

          Show
          cmccabe Colin P. McCabe added a comment - after some consultation with the Maven experts, it seems that including a Maven plugin in the same repository you use it in is not supported. I think the way forward here is to maintain cmake-maven-ng as its own Apache project. In the meantime, I have put cmake-maven-ng on Apache Extras under http://code.google.com/a/apache-extras.org/p/cmake-maven-ng/ . Maven can make use of this repository in the meantime. The following patch implements this.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Here's a new version which fixes the problem with "mvn site".

          • It also adds the ability to recognize native test failures to test-patch.sh.
          • Also fix a bug with the test classpath that some native unit tests were getting.
          Show
          cmccabe Colin P. McCabe added a comment - Here's a new version which fixes the problem with "mvn site". It also adds the ability to recognize native test failures to test-patch.sh. Also fix a bug with the test classpath that some native unit tests were getting.
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1230/)
          reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
          HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml

          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1230 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1230/ ) reverting commit 1399898 ( HADOOP-8887 ) (Revision 1399972) HADOOP-8887 . Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1200/)
          reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
          HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml

          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1200 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1200/ ) reverting commit 1399898 ( HADOOP-8887 ) (Revision 1399972) HADOOP-8887 . Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-Yarn-trunk #8 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/8/)
          reverting commit 1399898 (HADOOP-8887) (Revision 1399972)
          HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml

          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml
          Show
          hudson Hudson added a comment - Integrated in Hadoop-Yarn-trunk #8 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/8/ ) reverting commit 1399898 ( HADOOP-8887 ) (Revision 1399972) HADOOP-8887 . Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #2893 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2893/)
          reverting commit 1399898 (HADOOP-8887) (Revision 1399972)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml
          Show
          hudson Hudson added a comment - Integrated in Hadoop-trunk-Commit #2893 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2893/ ) reverting commit 1399898 ( HADOOP-8887 ) (Revision 1399972) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399972 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          I've just reverted both branch-2 and trunk, while the dist tarball is being built correctly, as Andrew pointed out site is not working, we have to fix that.

          Show
          tucu00 Alejandro Abdelnur added a comment - I've just reverted both branch-2 and trunk, while the dist tarball is being built correctly, as Andrew pointed out site is not working, we have to fix that.
          Hide
          sureshms Suresh Srinivas added a comment -

          @Alejandro no worries.

          Show
          sureshms Suresh Srinivas added a comment - @Alejandro no worries.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Committed to branch-2 after tweaking POM versions, did a full build and verified all generated JARs are the correct version. Apologies for the noise.

          Show
          tucu00 Alejandro Abdelnur added a comment - Committed to branch-2 after tweaking POM versions, did a full build and verified all generated JARs are the correct version. Apologies for the noise.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Thanks Suresh, my bad, missed to tweak the POM versions when merging, I'll do it again (this time properly). Nicholas, as Colin indicated, javadoc warnings are taken care by the patch, just it does not show until the patch is committed (one of these days I'll try to tweak test-patch to do what we do in Oozie, we run javadoc twice, once without the patch and once with the patch, and we compare the warning numbers, less or equal good).

          Show
          tucu00 Alejandro Abdelnur added a comment - Thanks Suresh, my bad, missed to tweak the POM versions when merging, I'll do it again (this time properly). Nicholas, as Colin indicated, javadoc warnings are taken care by the patch, just it does not show until the patch is committed (one of these days I'll try to tweak test-patch to do what we do in Oozie, we run javadoc twice, once without the patch and once with the patch, and we compare the warning numbers, less or equal good).
          Hide
          szetszwo Tsz Wo Nicholas Sze added a comment -

          How about the javadoc warnings?

          Show
          szetszwo Tsz Wo Nicholas Sze added a comment - How about the javadoc warnings?
          Hide
          sureshms Suresh Srinivas added a comment -

          I reverted the branch-2 patch. Some build issues and version number issues have been identified with that merge. Perhaps you need a separate patch for branch-2.

          Show
          sureshms Suresh Srinivas added a comment - I reverted the branch-2 patch. Some build issues and version number issues have been identified with that merge. Perhaps you need a separate patch for branch-2.
          Hide
          hudson Hudson added a comment -

          Integrated in Hadoop-trunk-Commit #2889 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2889/)
          HADOOP-8887. Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898
          Files :

          • /hadoop/common/trunk/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java
          • /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java
          • /hadoop/common/trunk/dev-support/pom.xml
          • /hadoop/common/trunk/dev-support/test-patch.sh
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml
          • /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c
          • /hadoop/common/trunk/pom.xml
          Show
          hudson Hudson added a comment - Integrated in Hadoop-trunk-Commit #2889 (See https://builds.apache.org/job/Hadoop-trunk-Commit/2889/ ) HADOOP-8887 . Use a Maven plugin to build the native code using CMake. (cmccabe via tucu) (Revision 1399898) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1399898 Files : /hadoop/common/trunk/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/dev-support/findbugsExcludeFile.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/pom.xml /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/CompileMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/TestMojo.java /hadoop/common/trunk/dev-support/cmake-maven-ng-plugin/src/main/java/org/apache/hadoop/cmake/maven/ng/Utils.java /hadoop/common/trunk/dev-support/pom.xml /hadoop/common/trunk/dev-support/test-patch.sh /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-tools/hadoop-pipes/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/pom.xml /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c /hadoop/common/trunk/pom.xml
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          Thanks Colin. Committed to trunk and branch-2.

          Show
          tucu00 Alejandro Abdelnur added a comment - Thanks Colin. Committed to trunk and branch-2.
          Hide
          cmccabe Colin P. McCabe added a comment -

          test failures are unrelated

          Show
          cmccabe Colin P. McCabe added a comment - test failures are unrelated
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12549601/HADOOP-8887.006.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 4 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:

          org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
          org.apache.hadoop.hdfs.server.balancer.TestBalancer

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1642//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1642//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12549601/HADOOP-8887.006.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 4 warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: org.apache.hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits org.apache.hadoop.hdfs.server.balancer.TestBalancer +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1642//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1642//console This message is automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          This new version should fix the javac warnings.

          The javadoc warnings will be suppressed when the patch is actually checked in (I bumped OK_JAVADOC_WARNINGS to 14). The source of the Javadoc warnings is that we're getting warnings about a few annotations which are necessary for Maven plugins (It warns about "@goal" and "@phase", but Maven needs these).

          Show
          cmccabe Colin P. McCabe added a comment - This new version should fix the javac warnings. The javadoc warnings will be suppressed when the patch is actually checked in (I bumped OK_JAVADOC_WARNINGS to 14). The source of the Javadoc warnings is that we're getting warnings about a few annotations which are necessary for Maven plugins (It warns about "@goal" and "@phase", but Maven needs these).
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548991/HADOOP-8887.005.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          -1 javac. The applied patch generated 2031 javac compiler warnings (more than the trunk's current 2030 warnings).

          -1 javadoc. The javadoc tool appears to have generated 4 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548991/HADOOP-8887.005.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. -1 javac . The applied patch generated 2031 javac compiler warnings (more than the trunk's current 2030 warnings). -1 javadoc . The javadoc tool appears to have generated 4 warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1641//console This message is automatically generated.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          +1 pending jenkins.

          Show
          tucu00 Alejandro Abdelnur added a comment - +1 pending jenkins.
          Hide
          cmccabe Colin P. McCabe added a comment -

          oh, also, I added the default parameters as tucu suggested.

          Show
          cmccabe Colin P. McCabe added a comment - oh, also, I added the default parameters as tucu suggested.
          Hide
          cmccabe Colin P. McCabe added a comment -

          By the way, one thing that's cool about this patch is that you can now run

          mvn test -Pnative -Dtest=test_native_mini_dfs
          

          and it will run the native test, just like you would expect.

          Show
          cmccabe Colin P. McCabe added a comment - By the way, one thing that's cool about this patch is that you can now run mvn test -Pnative -Dtest=test_native_mini_dfs and it will run the native test, just like you would expect.
          Hide
          cmccabe Colin P. McCabe added a comment -
          • move org.apache.maven.plugin.cmake.ng.* to org.apache.hadoop.cmake.maven.ng.*
          • removed CleanMojo
          • merged GenerateMojo and CompileMojo
          • clearer test output.
          Show
          cmccabe Colin P. McCabe added a comment - move org.apache.maven.plugin.cmake.ng.* to org.apache.hadoop.cmake.maven.ng.* removed CleanMojo merged GenerateMojo and CompileMojo clearer test output.
          Hide
          tomwhite Tom White added a comment -

          Does it make sense to put it in Hadoop for now, and then spin it off into another project later?

          If we do that we should at least put it in a org.apache.hadoop package since the Hadoop PMC doesn't control the org.apache.maven namespace.

          Show
          tomwhite Tom White added a comment - Does it make sense to put it in Hadoop for now, and then spin it off into another project later? If we do that we should at least put it in a org.apache.hadoop package since the Hadoop PMC doesn't control the org.apache.maven namespace.
          Hide
          cmccabe Colin P. McCabe added a comment -

          does compiled test stuff need to be segregated in target/test-native?

          No.

          Show
          cmccabe Colin P. McCabe added a comment - does compiled test stuff need to be segregated in target/test-native? No.
          Hide
          tucu00 Alejandro Abdelnur added a comment -

          got it, ok on clean. on the defaults, make sure you have tests for them.

          You've missed one question from prev comment: where test code lives? does compiled test stuff need to be segregated in target/test-native?

          Show
          tucu00 Alejandro Abdelnur added a comment - got it, ok on clean. on the defaults, make sure you have tests for them. You've missed one question from prev comment: where test code lives? does compiled test stuff need to be segregated in target/test-native?
          Hide
          cmccabe Colin P. McCabe added a comment -

          [defaults]

          I'll add defaults for those parameters.

          CleanMojo for windows, I don't see how, but if that is the case we should introduce it then if needed, not now just in case.

          My big concern is just that people will put their output directories somewhere other than in a directory that gets auto-cleaned. I think if we skip the clean step, we have to somehow enforce the constraint that the output directory gets cleared by mvn clean; otherwise we're just creating hidden dangers for the future.

          generate/make, I'd merge them in a single Mojo as there are not usecases for using them apart, plus generate does not generate sources but make files.

          OK. I'm fine with merging the cmake and make steps. It probably will make the pom.xml files shorter, which we can always use.

          I have not run the patch yet, but I assume that ALL generated/created files are done under output and the source tree is not polluted with files created at build time.

          Yep.

          Show
          cmccabe Colin P. McCabe added a comment - [defaults] I'll add defaults for those parameters. CleanMojo for windows, I don't see how, but if that is the case we should introduce it then if needed, not now just in case. My big concern is just that people will put their output directories somewhere other than in a directory that gets auto-cleaned. I think if we skip the clean step, we have to somehow enforce the constraint that the output directory gets cleared by mvn clean; otherwise we're just creating hidden dangers for the future. generate/make, I'd merge them in a single Mojo as there are not usecases for using them apart, plus generate does not generate sources but make files. OK. I'm fine with merging the cmake and make steps. It probably will make the pom.xml files shorter, which we can always use. I have not run the patch yet, but I assume that ALL generated/created files are done under output and the source tree is not polluted with files created at build time. Yep.
          Hide
          tucu00 Alejandro Abdelnur added a comment -
          • Unless I'm missing something, I don't see patch anything like the following to define defaults:
               * @parameter  expression="${source}" default-value="${basedir}/src/main/native"
               * @parameter  expression="${output}" default-value="${project.build.directory}/native"
          
          • CleanMojo for windows, I don't see how, but if that is the case we should introduce it then if needed, not now just in case.
          • generate/make, I'd merge them in a single Mojo as there are not usecases for using them apart, plus generate does not generate sources but make files.
          • where test code lives? does compiled test stuff need to be segregated in target/test-native?

          As an additional reason for removing clean & merging generate/make is that the amount of plugins config in the POMs are reduced from 3 to 1.

          I have not run the patch yet, but I assume that ALL generated/created files are done under output and the source tree is not polluted with files created at build time.

          Show
          tucu00 Alejandro Abdelnur added a comment - Unless I'm missing something, I don't see patch anything like the following to define defaults: * @parameter expression= "${source}" default -value= "${basedir}/src/main/ native " * @parameter expression= "${output}" default -value= "${project.build.directory}/ native " CleanMojo for windows, I don't see how, but if that is the case we should introduce it then if needed, not now just in case. generate/make, I'd merge them in a single Mojo as there are not usecases for using them apart, plus generate does not generate sources but make files. where test code lives? does compiled test stuff need to be segregated in target/test-native? As an additional reason for removing clean & merging generate/make is that the amount of plugins config in the POMs are reduced from 3 to 1. I have not run the patch yet, but I assume that ALL generated/created files are done under output and the source tree is not polluted with files created at build time.
          Hide
          cmccabe Colin P. McCabe added a comment -

          plugin root package should be org.apache.hadoop if in Hadoop.

          OK.

          source directory should be settable via an 'source' property which defaults to ${basedir}/src/main/native

          It is, via GenerateMojo#source

          build directory should be settable via an 'outputDirectory' property which defaults to ${project.build.directory}/native if not set.

          It is, via GenerateMojo#output and CompileMojo#output.

          what is the diff between the output and target params in the CompileMojo? Do we need both? see prev comment on naming.

          "Build target" would be something like Debug, Production, etc. "output" is a
          directory. I will add a comment explaining this.

          CleanMojo, why do we need this one? 'mvn clean' already takes care of it.

          'mvn clean' will delete the 'target' directory, but we don't enforce the concept that the CMake-ng output directory is inside that directory. We could enforce this, and then make get rid of the clean target? However, we also might need this for when we're supporting Windows, maybe?

          what is the diff between cmake-generate and cmake-compile? Do we need 2 different Mojos? Do we gain something from it?

          cmake-generate runs the cmake application to create the Makefiles. cmake-compile actually runs Make on these generated files. It seems natural to separate these two steps. However, I don't have a specific reason why it has to be implemented this way – we could combine both steps into one. I was trying to go with the spirit of Maven, which separates code generation and compilation.

          Show
          cmccabe Colin P. McCabe added a comment - plugin root package should be org.apache.hadoop if in Hadoop. OK. source directory should be settable via an 'source' property which defaults to ${basedir}/src/main/native It is, via GenerateMojo#source build directory should be settable via an 'outputDirectory' property which defaults to ${project.build.directory}/native if not set. It is, via GenerateMojo#output and CompileMojo#output. what is the diff between the output and target params in the CompileMojo? Do we need both? see prev comment on naming. "Build target" would be something like Debug, Production, etc. "output" is a directory. I will add a comment explaining this. CleanMojo, why do we need this one? 'mvn clean' already takes care of it. 'mvn clean' will delete the 'target' directory, but we don't enforce the concept that the CMake-ng output directory is inside that directory. We could enforce this, and then make get rid of the clean target? However, we also might need this for when we're supporting Windows, maybe? what is the diff between cmake-generate and cmake-compile? Do we need 2 different Mojos? Do we gain something from it? cmake-generate runs the cmake application to create the Makefiles. cmake-compile actually runs Make on these generated files. It seems natural to separate these two steps. However, I don't have a specific reason why it has to be implemented this way – we could combine both steps into one. I was trying to go with the spirit of Maven, which separates code generation and compilation.
          Hide
          tucu00 Alejandro Abdelnur added a comment -
          • plugin root package should be org.apache.hadoop if in Hadoop.
          • source directory should be settable via an 'source' property which defaults to $ {basedir}

            /src/main/native

          • build directory should be settable via an 'outputDirectory' property which defaults to $ {project.build.directory}

            /native if not set.

          • what is the diff between the output and target params in the CompileMojo? Do we need both? see prev comment on naming.
          • CleanMojo, why do we need this one? 'mvn clean' already takes care of it.
          • what is the diff between cmake-generate and cmake-compile? Do we need 2 different Mojos? Do we gain something from it?
          Show
          tucu00 Alejandro Abdelnur added a comment - plugin root package should be org.apache.hadoop if in Hadoop. source directory should be settable via an 'source' property which defaults to $ {basedir} /src/main/native build directory should be settable via an 'outputDirectory' property which defaults to $ {project.build.directory} /native if not set. what is the diff between the output and target params in the CompileMojo? Do we need both? see prev comment on naming. CleanMojo, why do we need this one? 'mvn clean' already takes care of it. what is the diff between cmake-generate and cmake-compile? Do we need 2 different Mojos? Do we gain something from it?
          Hide
          cmccabe Colin P. McCabe added a comment -

          It would be nice to spin it off as a separate project, but that seems like it would just make it more difficult to iterate on the design in the short term, which is what we will need to get it truly polished. If it's a separate project, we'll have backwards compatibility constraints, and also need two changes (one to add a feature to it, another to update Hadoop to use the new version and features) for every change.

          Does it make sense to put it in Hadoop for now, and then spin it off into another project later?

          Show
          cmccabe Colin P. McCabe added a comment - It would be nice to spin it off as a separate project, but that seems like it would just make it more difficult to iterate on the design in the short term, which is what we will need to get it truly polished. If it's a separate project, we'll have backwards compatibility constraints, and also need two changes (one to add a feature to it, another to update Hadoop to use the new version and features) for every change. Does it make sense to put it in Hadoop for now, and then spin it off into another project later?
          Hide
          tomwhite Tom White added a comment -

          This plugin looks general purpose and it's in the org.apache.maven package, so perhaps it should go in the Maven Plugins project?

          Show
          tomwhite Tom White added a comment - This plugin looks general purpose and it's in the org.apache.maven package, so perhaps it should go in the Maven Plugins project?
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548345/HADOOP-8887.004.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 javadoc. The javadoc tool appears to have generated 8 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1584//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1584//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548345/HADOOP-8887.004.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 javadoc . The javadoc tool appears to have generated 8 warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1584//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1584//console This message is automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Need to explicitly set Maven findbugs plugin version to avoid javac warnings.

          Show
          cmccabe Colin P. McCabe added a comment - Need to explicitly set Maven findbugs plugin version to avoid javac warnings.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548319/HADOOP-8887.003.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified test files.

          -1 javac. The applied patch generated 2061 javac compiler warnings (more than the trunk's current 2053 warnings).

          -1 javadoc. The javadoc tool appears to have generated 8 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548319/HADOOP-8887.003.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. -1 javac . The applied patch generated 2061 javac compiler warnings (more than the trunk's current 2053 warnings). -1 javadoc . The javadoc tool appears to have generated 8 warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1577//console This message is automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Clean up warnings.

          Show
          cmccabe Colin P. McCabe added a comment - Clean up warnings.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548065/HADOOP-8887.002.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          -1 javac. The applied patch generated 2057 javac compiler warnings (more than the trunk's current 2053 warnings).

          -1 javadoc. The javadoc tool appears to have generated 8 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 10 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//artifact/trunk/patchprocess/newPatchFindbugsWarningscmake-maven-ng-plugin.html
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548065/HADOOP-8887.002.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. -1 javac . The applied patch generated 2057 javac compiler warnings (more than the trunk's current 2053 warnings). -1 javadoc . The javadoc tool appears to have generated 8 warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 10 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//artifact/trunk/patchprocess/newPatchFindbugsWarningscmake-maven-ng-plugin.html Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1569//console This message is automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Get rid of some warnings (none of them were actual bugs.)

          Show
          cmccabe Colin P. McCabe added a comment - Get rid of some warnings (none of them were actual bugs.)
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12548020/HADOOP-8887.001.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 2 new or modified test files.

          -1 javac. The applied patch generated 2057 javac compiler warnings (more than the trunk's current 2053 warnings).

          -1 javadoc. The javadoc tool appears to have generated 8 warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          -1 findbugs. The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//artifact/trunk/patchprocess/newPatchFindbugsWarningscmake-maven-ng-plugin.html
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12548020/HADOOP-8887.001.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. -1 javac . The applied patch generated 2057 javac compiler warnings (more than the trunk's current 2053 warnings). -1 javadoc . The javadoc tool appears to have generated 8 warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. -1 findbugs . The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in dev-support/cmake-maven-ng-plugin hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//artifact/trunk/patchprocess/newPatchFindbugsWarningscmake-maven-ng-plugin.html Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1566//console This message is automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          A word of explanation about the test-container-executor.c change: it seems that when launched directly from Maven rather than from a shell, SIGQUIT starts off blocked, causing the test to fail. The change manually unblocks this signal-- always a good idea to do before you start using a signal.

          Show
          cmccabe Colin P. McCabe added a comment - A word of explanation about the test-container-executor.c change: it seems that when launched directly from Maven rather than from a shell, SIGQUIT starts off blocked, causing the test to fail. The change manually unblocks this signal-- always a good idea to do before you start using a signal.

            People

            • Assignee:
              cmccabe Colin P. McCabe
              Reporter:
              cmccabe Colin P. McCabe
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development