Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14670

Increase minimum cmake version for all platforms

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-beta1
    • Fix Version/s: 3.0.0-beta1
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      CMake v3.1.0 is now the minimum version required to build Apache Hadoop's native components.

      Description

      cmake 2.6 is old at this point and I'd be greatly surprised if anyone is actually using it or testing against it. It's probably time to upgrade to something approaching modern. Plus:

      • Mac OS X already requires 3.0
      • If HADOOP-14667 gets committed, Windows bumps to 3.1
      • There is special handling in at least one CMakeLists.txt for versions less than 3.1

      Given the last two points, I'd propose making the minimum 3.1, if not something higher due to compiler support for newer compilers across all platforms.

      1. HADOOP-14670.00.patch
        9 kB
        Allen Wittenauer
      2. HADOOP-14670.01.patch
        12 kB
        Allen Wittenauer
      3. HADOOP-14670.02.patch
        12 kB
        Allen Wittenauer
      4. HADOOP-14670.03.patch
        13 kB
        Allen Wittenauer

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          Opened HADOOP-14873

          Show
          aw Allen Wittenauer added a comment - Opened HADOOP-14873
          Hide
          leftnoteasy Wangda Tan added a comment -

          Sure, let's move discussion to a new JIRA for this.

          Show
          leftnoteasy Wangda Tan added a comment - Sure, let's move discussion to a new JIRA for this.
          Hide
          aw Allen Wittenauer added a comment -

          At this point, we need to open a new JIRA for this.

          Show
          aw Allen Wittenauer added a comment - At this point, we need to open a new JIRA for this.
          Hide
          aw Allen Wittenauer added a comment -

          ===
          test-gpu-module.cc
          ===

          Um, that's not getting built by a C compiler. So whoever added that code pretty much broke the universe by requiring a C++ compiler and not setting the appropriate flags.

          Show
          aw Allen Wittenauer added a comment - === test-gpu-module.cc === Um, that's not getting built by a C compiler. So whoever added that code pretty much broke the universe by requiring a C++ compiler and not setting the appropriate flags.
          Hide
          leftnoteasy Wangda Tan added a comment -

          I want to check if the macro takes effect or not, so I set it to old standard to see will it cause build failure or not. Not sure if I have any misunderstandings of the macro.

          Show
          leftnoteasy Wangda Tan added a comment - I want to check if the macro takes effect or not, so I set it to old standard to see will it cause build failure or not. Not sure if I have any misunderstandings of the macro.
          Hide
          aw Allen Wittenauer added a comment -

          set (CMAKE_C_STANDARD 90)

          What did you expect this to do?

          Show
          aw Allen Wittenauer added a comment - set (CMAKE_C_STANDARD 90) What did you expect this to do?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Allen Wittenauer / Chris Douglas,

          I just checked changes of this JIRA doesn't relate to issues I saw, I tried to revert this patch but issue is still the same.

          It seems the set (CMAKE_C_STANDARD) doesn't work for the nodemanager project.
          I hardcoded to change set (CMAKE_C_STANDARD 99) to set (CMAKE_C_STANDARD 90) in nodemanager project.

          I tried on two different environment:

          1) Centos 6, cmake version 3.1.0, gcc 4.4.7
          For both 99/90 standard, all fail.

          2) OSX v10.12.4, cmake version 3.5.2, cc = "Apple LLVM version 8.1.0 (clang-802.0.42)".
          For both 99/90 standard, all succeeded.

          At least for the for loop in gpu-module.c is C99 only:

          for (int i = 0; i < n_minor_devices_to_block; i++) {
             // ...
          }
          

          Could you share some thoughts?

          Thanks,

          Show
          leftnoteasy Wangda Tan added a comment - Allen Wittenauer / Chris Douglas , I just checked changes of this JIRA doesn't relate to issues I saw, I tried to revert this patch but issue is still the same. It seems the set (CMAKE_C_STANDARD) doesn't work for the nodemanager project. I hardcoded to change set (CMAKE_C_STANDARD 99) to set (CMAKE_C_STANDARD 90) in nodemanager project. I tried on two different environment: 1) Centos 6, cmake version 3.1.0, gcc 4.4.7 For both 99/90 standard, all fail. 2) OSX v10.12.4, cmake version 3.5.2, cc = "Apple LLVM version 8.1.0 (clang-802.0.42)". For both 99/90 standard, all succeeded. At least for the for loop in gpu-module.c is C99 only: for ( int i = 0; i < n_minor_devices_to_block; i++) { // ... } Could you share some thoughts? Thanks,
          Hide
          aw Allen Wittenauer added a comment -

          It's basically:

          if(CMAKE_VERSION VERSION_LESS 3.1)
            message( FATAL_ERROR "Your CMake version is so old, it ran on a VIC-20." )
          endif()
          

          or whatever.

          Show
          aw Allen Wittenauer added a comment - It's basically: if (CMAKE_VERSION VERSION_LESS 3.1) message( FATAL_ERROR "Your CMake version is so old, it ran on a VIC-20." ) endif() or whatever.
          Hide
          chris.douglas Chris Douglas added a comment -

          For simplicity of tracking, let's move this to a followup JIRA. Forcing failure for versions less than 3.1 will just add a few lines. No reason to add reverts or double-commits to the history.

          Show
          chris.douglas Chris Douglas added a comment - For simplicity of tracking, let's move this to a followup JIRA. Forcing failure for versions less than 3.1 will just add a few lines. No reason to add reverts or double-commits to the history.
          Hide
          leftnoteasy Wangda Tan added a comment -

          Thanks comments Allen Wittenauer/Chris Douglas, I will investigate on my local build environment to see what's the best option. Downgrading priority to major.

          Show
          leftnoteasy Wangda Tan added a comment - Thanks comments Allen Wittenauer / Chris Douglas , I will investigate on my local build environment to see what's the best option. Downgrading priority to major.
          Hide
          aw Allen Wittenauer added a comment -

          What C standard does cmake use if this is isn't fatal?

          Whatever the compiler defaults to.

          It might be hard for users to debug, if they don't get a clear error message using old tooling...

          Welcome to native compilation, regardless of the project.

          Would it be too pedantic to add an explicit failure directive? This could be a followup JIRA, though.

          A follow-up JIRA is the way to do this. It's like 4 lines of code in HadoopCommon.cmake. (I think everything imports that now.)

          Show
          aw Allen Wittenauer added a comment - What C standard does cmake use if this is isn't fatal? Whatever the compiler defaults to. It might be hard for users to debug, if they don't get a clear error message using old tooling... Welcome to native compilation, regardless of the project. Would it be too pedantic to add an explicit failure directive? This could be a followup JIRA, though. A follow-up JIRA is the way to do this. It's like 4 lines of code in HadoopCommon.cmake. (I think everything imports that now.)
          Hide
          chris.douglas Chris Douglas added a comment -

          What C standard does cmake use if this is isn't fatal? YARN-5719 was trying to make sure we picked something. It might be hard for users to debug, if they don't get a clear error message using old tooling...

          Would it be too pedantic to add an explicit failure directive? This could be a followup JIRA, though.

          Show
          chris.douglas Chris Douglas added a comment - What C standard does cmake use if this is isn't fatal? YARN-5719 was trying to make sure we picked something. It might be hard for users to debug, if they don't get a clear error message using old tooling... Would it be too pedantic to add an explicit failure directive? This could be a followup JIRA, though.
          Hide
          aw Allen Wittenauer added a comment -

          Given that the compile will fail either way, does it matter?

          Show
          aw Allen Wittenauer added a comment - Given that the compile will fail either way, does it matter?
          Hide
          leftnoteasy Wangda Tan added a comment -

          Allen Wittenauer, we just saw issue of building trunk includes this patch has compilation issue. (succeeded in centos 7 but failed in centos 7).

          According to https://cmake.org/cmake/help/v3.0/command/cmake_minimum_required.html, FATAL parameter ignored by cmake with newer version.

          Failure message looks like:

          [WARNING] .../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/modules/gpu/gpu-module.c:107: error: ‘for’ loop initial declarations are only allowed in C99 mode
          

          It looks like we need to either find a better way to fail build with lower versioned cmake, or we need to add set C99 macros back.

          Show
          leftnoteasy Wangda Tan added a comment - Allen Wittenauer , we just saw issue of building trunk includes this patch has compilation issue. (succeeded in centos 7 but failed in centos 7). According to https://cmake.org/cmake/help/v3.0/command/cmake_minimum_required.html , FATAL parameter ignored by cmake with newer version. Failure message looks like: [WARNING] .../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/ native /container-executor/impl/modules/gpu/gpu-module.c:107: error: ‘ for ’ loop initial declarations are only allowed in C99 mode It looks like we need to either find a better way to fail build with lower versioned cmake, or we need to add set C99 macros back.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12281 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12281/)
          HADOOP-14670. Increase minimum cmake version for all platforms (aw: rev 71bbb86d69ac474596f5619d22718e9f7ff5f9dc)

          • (edit) dev-support/docker/Dockerfile
          • (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/CMakeLists.txt
          • (edit) hadoop-common-project/hadoop-common/HadoopCommon.cmake
          • (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt
          • (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/CMakeLists.txt
          • (edit) start-build-env.sh
          • (edit) BUILDING.txt
          • (edit) hadoop-tools/hadoop-pipes/src/CMakeLists.txt
          • (edit) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/bzip2/org_apache_hadoop_io_compress_bzip2.h
          • (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          • (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • (edit) hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/CompileMojo.java
          • (edit) hadoop-common-project/hadoop-common/src/CMakeLists.txt
          • (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12281 (See https://builds.apache.org/job/Hadoop-trunk-Commit/12281/ ) HADOOP-14670 . Increase minimum cmake version for all platforms (aw: rev 71bbb86d69ac474596f5619d22718e9f7ff5f9dc) (edit) dev-support/docker/Dockerfile (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/CMakeLists.txt (edit) hadoop-common-project/hadoop-common/HadoopCommon.cmake (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/CMakeLists.txt (edit) start-build-env.sh (edit) BUILDING.txt (edit) hadoop-tools/hadoop-pipes/src/CMakeLists.txt (edit) hadoop-common-project/hadoop-common/src/main/native/src/org/apache/hadoop/io/compress/bzip2/org_apache_hadoop_io_compress_bzip2.h (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt (edit) hadoop-maven-plugins/src/main/java/org/apache/hadoop/maven/plugin/cmakebuilder/CompileMojo.java (edit) hadoop-common-project/hadoop-common/src/CMakeLists.txt (edit) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          Hide
          aw Allen Wittenauer added a comment -

          Thanks!

          Committing to trunk.

          Show
          aw Allen Wittenauer added a comment - Thanks! Committing to trunk.
          Hide
          chris.douglas Chris Douglas added a comment -

          Checked that this removed YARN-5719, which this makes redundant.

          +1 skimmed the patch and lgtm.

          Show
          chris.douglas Chris Douglas added a comment - Checked that this removed YARN-5719 , which this makes redundant. +1 skimmed the patch and lgtm.
          Hide
          aw Allen Wittenauer added a comment -

          HADOOP-14816 has nearly the same Dockerfile changes.

          Show
          aw Allen Wittenauer added a comment - HADOOP-14816 has nearly the same Dockerfile changes.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 15m 12s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 0m 28s Maven dependency ordering for branch
          +1 mvninstall 20m 36s trunk passed
          +1 compile 22m 22s trunk passed
          +1 checkstyle 2m 42s trunk passed
          +1 mvnsite 16m 55s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes .
          -1 findbugs 1m 21s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings.
          +1 javadoc 7m 37s trunk passed
                Patch Compile Tests
          0 mvndep 0m 24s Maven dependency ordering for patch
          +1 mvninstall 23m 55s the patch passed
          +1 compile 17m 36s the patch passed
          +1 cc 17m 36s the patch passed
          +1 javac 17m 36s the patch passed
          +1 checkstyle 2m 43s the patch passed
          +1 mvnsite 16m 56s the patch passed
          +1 shellcheck 0m 0s There were no new shellcheck issues.
          +1 shelldocs 0m 11s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes .
          +1 findbugs 6m 29s the patch passed
          +1 javadoc 7m 35s the patch passed
                Other Tests
          -1 unit 18m 12s root in the patch failed.
          +1 asflicense 0m 49s The patch does not generate ASF License warnings.
          208m 32s



          Reason Tests
          Failed junit tests hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem
            hadoop.net.TestDNS
            hadoop.security.TestRaceWhenRelogin



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14670
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884512/HADOOP-14670.03.patch
          Optional Tests asflicense shellcheck shelldocs compile cc mvnsite javac unit javadoc mvninstall findbugs checkstyle
          uname Linux 0359caf5ba84 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fd66a24
          Default Java 1.8.0_144
          shellcheck v0.4.6
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/testReport/
          modules C: hadoop-maven-plugins hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 12s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 20m 36s trunk passed +1 compile 22m 22s trunk passed +1 checkstyle 2m 42s trunk passed +1 mvnsite 16m 55s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes . -1 findbugs 1m 21s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. +1 javadoc 7m 37s trunk passed       Patch Compile Tests 0 mvndep 0m 24s Maven dependency ordering for patch +1 mvninstall 23m 55s the patch passed +1 compile 17m 36s the patch passed +1 cc 17m 36s the patch passed +1 javac 17m 36s the patch passed +1 checkstyle 2m 43s the patch passed +1 mvnsite 16m 56s the patch passed +1 shellcheck 0m 0s There were no new shellcheck issues. +1 shelldocs 0m 11s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes . +1 findbugs 6m 29s the patch passed +1 javadoc 7m 35s the patch passed       Other Tests -1 unit 18m 12s root in the patch failed. +1 asflicense 0m 49s The patch does not generate ASF License warnings. 208m 32s Reason Tests Failed junit tests hadoop.fs.viewfs.TestViewFileSystemWithAuthorityLocalFileSystem   hadoop.net.TestDNS   hadoop.security.TestRaceWhenRelogin Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14670 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884512/HADOOP-14670.03.patch Optional Tests asflicense shellcheck shelldocs compile cc mvnsite javac unit javadoc mvninstall findbugs checkstyle uname Linux 0359caf5ba84 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fd66a24 Default Java 1.8.0_144 shellcheck v0.4.6 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/testReport/ modules C: hadoop-maven-plugins hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 25m 59s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 0m 22s Maven dependency ordering for branch
          +1 mvninstall 20m 26s trunk passed
          +1 compile 22m 19s trunk passed
          +1 checkstyle 2m 43s trunk passed
          +1 mvnsite 16m 59s trunk passed
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes .
          -1 findbugs 1m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings.
          +1 javadoc 7m 37s trunk passed
                Patch Compile Tests
          0 mvndep 0m 27s Maven dependency ordering for patch
          +1 mvninstall 23m 40s the patch passed
          +1 compile 17m 32s the patch passed
          +1 cc 17m 32s the patch passed
          +1 javac 17m 32s the patch passed
          +1 checkstyle 2m 41s the patch passed
          +1 mvnsite 16m 52s the patch passed
          +1 shellcheck 0m 0s There were no new shellcheck issues.
          +1 shelldocs 0m 10s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
          0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes .
          +1 findbugs 6m 17s the patch passed
          +1 javadoc 7m 42s the patch passed
                Other Tests
          -1 unit 17m 50s root in the patch failed.
          +1 asflicense 0m 49s The patch does not generate ASF License warnings.
          218m 12s



          Reason Tests
          Failed junit tests hadoop.security.TestRaceWhenRelogin
            hadoop.ipc.TestIPC



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14670
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884512/HADOOP-14670.03.patch
          Optional Tests asflicense shellcheck shelldocs compile cc mvnsite javac unit javadoc mvninstall findbugs checkstyle
          uname Linux 2b7d27f51219 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / fd66a24
          Default Java 1.8.0_144
          shellcheck v0.4.6
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/testReport/
          modules C: hadoop-maven-plugins hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 25m 59s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 0m 22s Maven dependency ordering for branch +1 mvninstall 20m 26s trunk passed +1 compile 22m 19s trunk passed +1 checkstyle 2m 43s trunk passed +1 mvnsite 16m 59s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes . -1 findbugs 1m 23s hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. +1 javadoc 7m 37s trunk passed       Patch Compile Tests 0 mvndep 0m 27s Maven dependency ordering for patch +1 mvninstall 23m 40s the patch passed +1 compile 17m 32s the patch passed +1 cc 17m 32s the patch passed +1 javac 17m 32s the patch passed +1 checkstyle 2m 41s the patch passed +1 mvnsite 16m 52s the patch passed +1 shellcheck 0m 0s There were no new shellcheck issues. +1 shelldocs 0m 10s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-tools/hadoop-pipes . +1 findbugs 6m 17s the patch passed +1 javadoc 7m 42s the patch passed       Other Tests -1 unit 17m 50s root in the patch failed. +1 asflicense 0m 49s The patch does not generate ASF License warnings. 218m 12s Reason Tests Failed junit tests hadoop.security.TestRaceWhenRelogin   hadoop.ipc.TestIPC Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14670 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884512/HADOOP-14670.03.patch Optional Tests asflicense shellcheck shelldocs compile cc mvnsite javac unit javadoc mvninstall findbugs checkstyle uname Linux 2b7d27f51219 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / fd66a24 Default Java 1.8.0_144 shellcheck v0.4.6 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/testReport/ modules C: hadoop-maven-plugins hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 16m 35s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 0m 20s Maven dependency ordering for branch
          +1 mvninstall 13m 38s trunk passed
          +1 compile 14m 52s trunk passed
          +1 mvnsite 11m 22s trunk passed
                Patch Compile Tests
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 16m 31s the patch passed
          +1 compile 11m 52s the patch passed
          +1 cc 11m 52s the patch passed
          +1 javac 11m 52s the patch passed
          +1 mvnsite 11m 22s the patch passed
          +1 shellcheck 0m 0s There were no new shellcheck issues.
          +1 shelldocs 0m 9s There were no new shelldocs issues.
          +1 whitespace 0m 0s The patch has no whitespace issues.
                Other Tests
          -1 unit 14m 22s root in the patch failed.
          +1 asflicense 0m 35s The patch does not generate ASF License warnings.
          112m 36s



          Reason Tests
          Failed junit tests hadoop.security.TestKDiag



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14670
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884507/HADOOP-14670.01.patch
          Optional Tests asflicense shellcheck shelldocs compile cc mvnsite javac unit
          uname Linux bc58d494d31c 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / a20e710
          Default Java 1.8.0_144
          shellcheck v0.4.6
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 35s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 0m 20s Maven dependency ordering for branch +1 mvninstall 13m 38s trunk passed +1 compile 14m 52s trunk passed +1 mvnsite 11m 22s trunk passed       Patch Compile Tests 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 16m 31s the patch passed +1 compile 11m 52s the patch passed +1 cc 11m 52s the patch passed +1 javac 11m 52s the patch passed +1 mvnsite 11m 22s the patch passed +1 shellcheck 0m 0s There were no new shellcheck issues. +1 shelldocs 0m 9s There were no new shelldocs issues. +1 whitespace 0m 0s The patch has no whitespace issues.       Other Tests -1 unit 14m 22s root in the patch failed. +1 asflicense 0m 35s The patch does not generate ASF License warnings. 112m 36s Reason Tests Failed junit tests hadoop.security.TestKDiag Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14670 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884507/HADOOP-14670.01.patch Optional Tests asflicense shellcheck shelldocs compile cc mvnsite javac unit uname Linux bc58d494d31c 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7 11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a20e710 Default Java 1.8.0_144 shellcheck v0.4.6 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          A patch to the testing environment has been detected.
          Re-executing against the patched versions to perform further tests.
          The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/console in case of problems.

          Show
          hadoopqa Hadoop QA added a comment - A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/13137/console in case of problems.
          Hide
          aw Allen Wittenauer added a comment -

          -03:

          • correct patch file
          Show
          aw Allen Wittenauer added a comment - -03: correct patch file
          Hide
          hadoopqa Hadoop QA added a comment -

          A patch to the testing environment has been detected.
          Re-executing against the patched versions to perform further tests.
          The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/console in case of problems.

          Show
          hadoopqa Hadoop QA added a comment - A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/13136/console in case of problems.
          Hide
          aw Allen Wittenauer added a comment -

          -02:

          • we can remove the double make in the maven plugin now
          Show
          aw Allen Wittenauer added a comment - -02: we can remove the double make in the maven plugin now
          Hide
          hadoopqa Hadoop QA added a comment -

          A patch to the testing environment has been detected.
          Re-executing against the patched versions to perform further tests.
          The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/console in case of problems.

          Show
          hadoopqa Hadoop QA added a comment - A patch to the testing environment has been detected. Re-executing against the patched versions to perform further tests. The console is at https://builds.apache.org/job/PreCommit-HADOOP-Build/13135/console in case of problems.
          Hide
          aw Allen Wittenauer added a comment -

          -01:

          • update cmake in the dockerfile directly from cmake.org rather than wait for a Xenial Dockerfile
          • fix up some clang issues, because clang is not gcc.
          • add ability to drop to docker's root from within the start-build-env.sh environment to ease debugging packaging issues
          Show
          aw Allen Wittenauer added a comment - -01: update cmake in the dockerfile directly from cmake.org rather than wait for a Xenial Dockerfile fix up some clang issues, because clang is not gcc. add ability to drop to docker's root from within the start-build-env.sh environment to ease debugging packaging issues
          Hide
          stevel@apache.org Steve Loughran added a comment -

          update seems a reasonable idea, but looks like yetus is still on the older versions

          [WARNING] CMake Error at CMakeLists.txt:23 (cmake_minimum_required):
          [WARNING]   CMake 3.1 or higher is required.  You are running version 2.8.12.2
          
          Show
          stevel@apache.org Steve Loughran added a comment - update seems a reasonable idea, but looks like yetus is still on the older versions [WARNING] CMake Error at CMakeLists.txt:23 (cmake_minimum_required): [WARNING] CMake 3.1 or higher is required. You are running version 2.8.12.2
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
                Prechecks
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
                trunk Compile Tests
          0 mvndep 1m 41s Maven dependency ordering for branch
          +1 mvninstall 14m 9s trunk passed
          +1 compile 18m 24s trunk passed
          -1 mvnsite 1m 53s root in trunk failed.
                Patch Compile Tests
          0 mvndep 0m 15s Maven dependency ordering for patch
          +1 mvninstall 17m 9s the patch passed
          -1 compile 0m 50s root in the patch failed.
          -1 cc 0m 50s root in the patch failed.
          -1 javac 0m 50s root in the patch failed.
          -1 mvnsite 1m 37s root in the patch failed.
          +1 whitespace 0m 0s The patch has no whitespace issues.
                Other Tests
          -1 unit 8m 22s root in the patch failed.
          +1 asflicense 0m 32s The patch does not generate ASF License warnings.
          65m 53s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:14b5c93
          JIRA Issue HADOOP-14670
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884315/HADOOP-14670.00.patch
          Optional Tests asflicense compile cc mvnsite javac unit
          uname Linux 1d7e64476e3a 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 8201ed8
          Default Java 1.8.0_144
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/branch-mvnsite-root.txt
          compile https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-compile-root.txt
          cc https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-compile-root.txt
          javac https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-compile-root.txt
          mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-mvnsite-root.txt
          unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-unit-root.txt
          Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: .
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/console
          Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.       trunk Compile Tests 0 mvndep 1m 41s Maven dependency ordering for branch +1 mvninstall 14m 9s trunk passed +1 compile 18m 24s trunk passed -1 mvnsite 1m 53s root in trunk failed.       Patch Compile Tests 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 17m 9s the patch passed -1 compile 0m 50s root in the patch failed. -1 cc 0m 50s root in the patch failed. -1 javac 0m 50s root in the patch failed. -1 mvnsite 1m 37s root in the patch failed. +1 whitespace 0m 0s The patch has no whitespace issues.       Other Tests -1 unit 8m 22s root in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 65m 53s Subsystem Report/Notes Docker Image:yetus/hadoop:14b5c93 JIRA Issue HADOOP-14670 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12884315/HADOOP-14670.00.patch Optional Tests asflicense compile cc mvnsite javac unit uname Linux 1d7e64476e3a 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8201ed8 Default Java 1.8.0_144 mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/branch-mvnsite-root.txt compile https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-compile-root.txt cc https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-compile-root.txt javac https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-compile-root.txt mvnsite https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-mvnsite-root.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/artifact/patchprocess/patch-unit-root.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-native-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask hadoop-tools/hadoop-pipes . U: . Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/13127/console Powered by Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

            People

            • Assignee:
              aw Allen Wittenauer
              Reporter:
              aw Allen Wittenauer
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development