Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9369

Use ctest to run tests for hadoop-hdfs-native-client

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently we write special rules in pom.xml to run tests in hadoop-hdfs-native-client. This jira proposes to run these tests using ctest to simplify pom.xml and improve portability.

        Issue Links

          Activity

          Hide
          aw Allen Wittenauer added a comment -

          We need an option (or just force it) to send ctest output to a file for later post-processing similar to what we do for bats+TAP.

          Show
          aw Allen Wittenauer added a comment - We need an option (or just force it) to send ctest output to a file for later post-processing similar to what we do for bats+TAP.
          Hide
          wheat9 Haohui Mai added a comment -

          It's okay to redirect the output to a file.

          Allen Wittenauer, do you have concrete ideas on which file that ctest should store its output to in order to be integrated with Yetus?

          Show
          wheat9 Haohui Mai added a comment - It's okay to redirect the output to a file. Allen Wittenauer , do you have concrete ideas on which file that ctest should store its output to in order to be integrated with Yetus?
          Hide
          aw Allen Wittenauer added a comment -

          The output should be dumped into a directory in target. We haven't written a ctest test format plug-in yet, but I suspect it will work like TAP does: give the plug-in the directory where ctest output lives and it'll get processed as appropriate.

          Show
          aw Allen Wittenauer added a comment - The output should be dumped into a directory in target. We haven't written a ctest test format plug-in yet, but I suspect it will work like TAP does: give the plug-in the directory where ctest output lives and it'll get processed as appropriate.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 4m 4s trunk passed
          +1 compile 0m 33s trunk passed with JDK v1.8.0_60
          +1 compile 0m 30s trunk passed with JDK v1.7.0_79
          +1 mvneclipse 0m 15s trunk passed
          +1 javadoc 0m 14s trunk passed with JDK v1.8.0_60
          +1 javadoc 0m 13s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 15s the patch passed
          +1 compile 0m 29s the patch passed with JDK v1.8.0_60
          +1 cc 0m 29s the patch passed
          +1 javac 0m 29s the patch passed
          +1 compile 0m 30s the patch passed with JDK v1.7.0_79
          +1 cc 0m 30s the patch passed
          +1 javac 0m 30s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 javadoc 0m 13s the patch passed with JDK v1.8.0_60
          +1 javadoc 0m 13s the patch passed with JDK v1.7.0_79
          +1 unit 0m 56s hadoop-hdfs-native-client in the patch passed with JDK v1.8.0_60.
          +1 unit 0m 53s hadoop-hdfs-native-client in the patch passed with JDK v1.7.0_79.
          +1 asflicense 0m 32s Patch does not generate ASF License warnings.
          13m 24s



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770407/HDFS-9369.000.patch
          JIRA Issue HDFS-9369
          Optional Tests asflicense javac javadoc mvninstall unit xml compile cc
          uname Linux eee2e9392894 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh
          git revision trunk / dac0463
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13366/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13366/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 4m 4s trunk passed +1 compile 0m 33s trunk passed with JDK v1.8.0_60 +1 compile 0m 30s trunk passed with JDK v1.7.0_79 +1 mvneclipse 0m 15s trunk passed +1 javadoc 0m 14s trunk passed with JDK v1.8.0_60 +1 javadoc 0m 13s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 15s the patch passed +1 compile 0m 29s the patch passed with JDK v1.8.0_60 +1 cc 0m 29s the patch passed +1 javac 0m 29s the patch passed +1 compile 0m 30s the patch passed with JDK v1.7.0_79 +1 cc 0m 30s the patch passed +1 javac 0m 30s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 javadoc 0m 13s the patch passed with JDK v1.8.0_60 +1 javadoc 0m 13s the patch passed with JDK v1.7.0_79 +1 unit 0m 56s hadoop-hdfs-native-client in the patch passed with JDK v1.8.0_60. +1 unit 0m 53s hadoop-hdfs-native-client in the patch passed with JDK v1.7.0_79. +1 asflicense 0m 32s Patch does not generate ASF License warnings. 13m 24s Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770407/HDFS-9369.000.patch JIRA Issue HDFS-9369 Optional Tests asflicense javac javadoc mvninstall unit xml compile cc uname Linux eee2e9392894 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh git revision trunk / dac0463 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13366/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-native-client U: hadoop-hdfs-project/hadoop-hdfs-native-client Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13366/console This message was automatically generated.
          Hide
          wheat9 Haohui Mai added a comment -

          It looks like there is no need to change the patch. The information is already available at target/Testing/Temporary/LastTest.log.

          Show
          wheat9 Haohui Mai added a comment - It looks like there is no need to change the patch. The information is already available at target/Testing/Temporary/LastTest.log .
          Hide
          jingzhao Jing Zhao added a comment -

          The change looks good to me. +1

          Show
          jingzhao Jing Zhao added a comment - The change looks good to me. +1
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8788 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8788/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8788 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8788/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          Hide
          wheat9 Haohui Mai added a comment -

          I've committed the patch to trunk and branch-2. Thanks for the reviews.

          Show
          wheat9 Haohui Mai added a comment - I've committed the patch to trunk and branch-2. Thanks for the reviews.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #665 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/665/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #665 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/665/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1389 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1389/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1389 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1389/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2532 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2532/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2532 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2532/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #654 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/654/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #654 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/654/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2594 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2594/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2594 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2594/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #594 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/594/)
          HDFS-9369. Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f)

          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #594 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/594/ ) HDFS-9369 . Use ctest to run tests for hadoop-hdfs-native-client. (wheat9: rev a06e6b81e08d6a6df62334cf51232e141861025f) hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs-tests/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/pom.xml hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfs/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs-native-client/src/CMakeLists.txt hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          cnauroth Chris Nauroth added a comment -

          The transition to ctest broke the Windows build and native test runs. I filed a pull request on HDFS-9459 to fix it.

          Show
          cnauroth Chris Nauroth added a comment - The transition to ctest broke the Windows build and native test runs. I filed a pull request on HDFS-9459 to fix it.

            People

            • Assignee:
              wheat9 Haohui Mai
              Reporter:
              wheat9 Haohui Mai
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development