Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Same as HADOOP-6671 for hdfs

      1. HDFS-2096-AJ.sh
        5 kB
        Alejandro Abdelnur
      2. HDFS-2096-AJ.patch
        186 kB
        Alejandro Abdelnur
      3. HDFS-2096-AI.sh
        5 kB
        Alejandro Abdelnur
      4. HDFS-2096-AI.patch
        186 kB
        Alejandro Abdelnur
      5. HDFS-2096-AH.sh
        5 kB
        Alejandro Abdelnur
      6. HDFS-2096-AH.patch
        185 kB
        Alejandro Abdelnur
      7. HDFS-2096-AG.sh
        5 kB
        Alejandro Abdelnur
      8. HDFS-2096-AG.patch
        184 kB
        Alejandro Abdelnur
      9. HDFS-2096-AF.sh
        5 kB
        Alejandro Abdelnur
      10. HDFS-2096-AF.patch
        202 kB
        Alejandro Abdelnur
      11. HDFS-2096-AE.sh
        5 kB
        Alejandro Abdelnur
      12. HDFS-2096-AE.patch
        191 kB
        Alejandro Abdelnur
      13. HDFS-2096-AD.sh
        5 kB
        Alejandro Abdelnur
      14. HDFS-2096-AD.patch
        76 kB
        Alejandro Abdelnur
      15. HDFS-2096-AC.sh
        5 kB
        Alejandro Abdelnur
      16. HDFS-2096-AC.patch
        76 kB
        Alejandro Abdelnur
      17. HDFS-2096-AB.sh
        5 kB
        Alejandro Abdelnur
      18. HDFS-2096-AB.patch
        50 kB
        Alejandro Abdelnur
      19. HDFS-2096-AA.sh
        3 kB
        Alejandro Abdelnur
      20. HDFS-2096-AA.patch
        48 kB
        Alejandro Abdelnur

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          np, I will then put the WebHDFS apt doc under hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site for the moment.

          Show
          Tsz Wo Nicholas Sze added a comment - np, I will then put the WebHDFS apt doc under hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-site for the moment.
          Hide
          Alejandro Abdelnur added a comment -

          Sorry, I should have been a bit more clear, it has to be done, mvn site & APT is not wired in hdfs yet, it has to be done.

          Show
          Alejandro Abdelnur added a comment - Sorry, I should have been a bit more clear, it has to be done, mvn site & APT is not wired in hdfs yet, it has to be done.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          But the directory hadoop-hdfs-project/hadoop-hdfs/src/site is not found.

          Also, "mvn site" failed if it is executed under hadoop-hdfs-project/hadoop-hdfs.

          Show
          Tsz Wo Nicholas Sze added a comment - But the directory hadoop-hdfs-project/hadoop-hdfs/src/site is not found. Also, "mvn site" failed if it is executed under hadoop-hdfs-project/hadoop-hdfs.
          Hide
          Alejandro Abdelnur added a comment -

          same dir structure of hadoop-common-project/hadoop-auth APT (src/site/...) under hadoop-hdfs-project/hadoop-hdfs

          Show
          Alejandro Abdelnur added a comment - same dir structure of hadoop-common-project/hadoop-auth APT (src/site/...) under hadoop-hdfs-project/hadoop-hdfs
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Hi Alejandro,

          Where should the HDFS apt doc go?

          Show
          Tsz Wo Nicholas Sze added a comment - Hi Alejandro, Where should the HDFS apt doc go?
          Hide
          Tom White added a comment -

          I committed this. Thanks, Alejandro!

          Show
          Tom White added a comment - I committed this. Thanks, Alejandro!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #758 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/758/)
          HDFS-2096. Mavenization of hadoop-hdfs. Contributed by Alejandro Abdelnur.

          tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159702
          Files :

          • /hadoop/common/trunk/hadoop-hdfs/src/main/native
          • /hadoop/common/trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data15bytes
          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tar.xml
          • /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-template.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/hdfs-site.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/test-patch.properties
          • /hadoop/common/trunk/hadoop-hdfs/bin
          • /hadoop/common/trunk/hadoop-hdfs/src/test/all-tests
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data15bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/main/xsl
          • /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-instrumented-template.xml
          • /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/NameNode.launch
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/checkstyle.xml
          • /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/DataNode.launch
          • /hadoop/common/trunk/hadoop-hdfs/src/main/conf/hadoop-metrics2.properties
          • /hadoop/common/trunk/hadoop-hdfs
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-14-dfs-dir.tgz
          • /hadoop/common/trunk/hadoop-hdfs/src/test/hdfs/org
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/hadoop-22-dfs-dir.tgz
          • /hadoop/common/trunk/hadoop-hdfs/src/main/webapps/proto-hdfs-web.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java
          • /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-bintar.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/xsl/configuration.xsl
          • /hadoop/common/trunk/hadoop-hdfs/ivy.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/hadoop-policy.xml
          • /hadoop/common/trunk/hadoop-hdfs/ivy/ivysettings.xml
          • /hadoop/common/trunk/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs/dev-support
          • /hadoop/common/trunk/hadoop-hdfs/build.xml
          • /hadoop/common/trunk/hadoop-common/BUILDING.txt
          • /hadoop/common/trunk/hadoop-hdfs/src/main/packages
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/empty-file
          • /hadoop/common/trunk/hadoop-hdfs/src/test/log4j.properties
          • /hadoop/common/trunk/hadoop-hdfs/src/main/conf/configuration.xsl
          • /hadoop/common/trunk/hadoop-common/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/webapps
          • /hadoop/common/trunk/dev-support/smart-apply-patch.sh
          • /hadoop/common/trunk/BUILDING.txt
          • /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/SpecificTestTemplate.launch
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hdfs-site.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/resources
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/checkstyle-noframes-sorted.xsl
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommission.java
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data120bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data1k
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/editsStored
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/hadoop-dfs-dir.txt
          • /hadoop/common/trunk/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/fsimageV18
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-policy.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/java/hdfs-default.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/fsimageV19
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-22-dfs-dir.tgz
          • /hadoop/common/trunk/hadoop-hdfs/ivy/libraries.properties
          • /hadoop/common/trunk/hdfs
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data60bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data60bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/java
          • /hadoop/common/trunk/hadoop-hdfs/src/main/conf/hdfs-site.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/native/c++/libhdfs
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources
          • /hadoop/common/trunk/hadoop-hdfs/src/main/conf/hdfs-site.xml.template
          • /hadoop/common/trunk/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml
          • /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/README.txt
          • /hadoop/common/trunk/hadoop-project/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-test-template.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/bin
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/log4j.properties
          • /hadoop/common/trunk/hadoop-hdfs/src/test/checkstyle.xml
          • /hadoop/common/trunk/hadoop-project-distro/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/commit-tests
          • /hadoop/common/trunk/hadoop-hdfs/src/main
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/editsStored.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/docs
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/smoke-tests
          • /hadoop/common/trunk/hadoop-hdfs/src/test/fi-site.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/java
          • /hadoop/common/trunk/hadoop-hdfs/src/main/conf
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/findbugsExcludeFile.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/editsStored
          • /hadoop/common/trunk/hadoop-hdfs/src/test/checkstyle-noframes-sorted.xsl
          • /hadoop/common/trunk/hadoop-hdfs/src/test/smoke-tests
          • /hadoop/common/trunk/hadoop-hdfs/src/c++
          • /hadoop/common/trunk/hadoop-hdfs/src/test/findbugsExcludeFile.xml
          • /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-instrumented-test-template.xml
          • /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/AllTests.launch
          • /hadoop/common/trunk/hadoop-project-distro
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data1k
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data120bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/fi-site.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/testHDFSConf.xml
          • /hadoop/common/trunk/hadoop-hdfs/lib/jdiff
          • /hadoop/common/trunk/hadoop-hdfs/src/main/docs
          • /hadoop/common/trunk/hadoop-project-distro/README.txt
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-dfs-dir.txt
          • /hadoop/common/trunk/hadoop-hdfs/conf
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org
          • /hadoop/common/trunk/hadoop-hdfs/src/test/krb5.conf
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/testHDFSConf.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data30bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/fsimageV18
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/fsimageV19
          • /hadoop/common/trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/HdfsProtos.java
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data30bytes
          • /hadoop/common/trunk/hadoop-hdfs/src/test/empty-file
          • /hadoop/common/trunk/hadoop-hdfs/src/main/webapps/proto-secondary-web.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/hadoop-14-dfs-dir.tgz
          • /hadoop/common/trunk/hadoop-hdfs/src/main/webapps/proto-datanode-web.xml
          • /hadoop/common/trunk/hadoop-hdfs/src/main/native/c++
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/jdiff
          • /hadoop/common/trunk/hadoop-hdfs/pom.xml
          • /hadoop/common/trunk/hadoop-hdfs/.gitignore
          • /hadoop/common/trunk/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/test-patch.properties
          • /hadoop/common/trunk/hadoop-hdfs/src/packages
          • /hadoop/common/trunk/hadoop-hdfs/src/test/commit-tests
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/editsStored.xml
          • /hadoop/common/trunk/hadoop-hdfs/dev-support/all-tests
          • /hadoop/common/trunk/hadoop-hdfs/src/main/webapps
          • /hadoop/common/trunk/hadoop-hdfs/src/test/resources/krb5.conf
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #758 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/758/ ) HDFS-2096 . Mavenization of hadoop-hdfs. Contributed by Alejandro Abdelnur. tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159702 Files : /hadoop/common/trunk/hadoop-hdfs/src/main/native /hadoop/common/trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/DataTransferProtos.java /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data15bytes /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-tar.xml /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-template.xml /hadoop/common/trunk/hadoop-hdfs/src/test/hdfs-site.xml /hadoop/common/trunk/hadoop-hdfs/src/test/test-patch.properties /hadoop/common/trunk/hadoop-hdfs/bin /hadoop/common/trunk/hadoop-hdfs/src/test/all-tests /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data15bytes /hadoop/common/trunk/hadoop-hdfs/src/main/xsl /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-instrumented-template.xml /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/NameNode.launch /hadoop/common/trunk/hadoop-hdfs/dev-support/checkstyle.xml /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/DataNode.launch /hadoop/common/trunk/hadoop-hdfs/src/main/conf/hadoop-metrics2.properties /hadoop/common/trunk/hadoop-hdfs /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-14-dfs-dir.tgz /hadoop/common/trunk/hadoop-hdfs/src/test/hdfs/org /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/hadoop-22-dfs-dir.tgz /hadoop/common/trunk/hadoop-hdfs/src/main/webapps/proto-hdfs-web.xml /hadoop/common/trunk/hadoop-hdfs/src/test/java /hadoop/common/trunk/hadoop-assemblies/src/main/resources/assemblies/hadoop-bintar.xml /hadoop/common/trunk/hadoop-hdfs/src/main/xsl/configuration.xsl /hadoop/common/trunk/hadoop-hdfs/ivy.xml /hadoop/common/trunk/hadoop-hdfs/src/test/hadoop-policy.xml /hadoop/common/trunk/hadoop-hdfs/ivy/ivysettings.xml /hadoop/common/trunk/pom.xml /hadoop/common/trunk/hadoop-hdfs/dev-support /hadoop/common/trunk/hadoop-hdfs/build.xml /hadoop/common/trunk/hadoop-common/BUILDING.txt /hadoop/common/trunk/hadoop-hdfs/src/main/packages /hadoop/common/trunk/hadoop-hdfs/src/test/resources/empty-file /hadoop/common/trunk/hadoop-hdfs/src/test/log4j.properties /hadoop/common/trunk/hadoop-hdfs/src/main/conf/configuration.xsl /hadoop/common/trunk/hadoop-common/pom.xml /hadoop/common/trunk/hadoop-hdfs/src/webapps /hadoop/common/trunk/dev-support/smart-apply-patch.sh /hadoop/common/trunk/BUILDING.txt /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/SpecificTestTemplate.launch /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hdfs-site.xml /hadoop/common/trunk/hadoop-hdfs/src/main/resources /hadoop/common/trunk/hadoop-hdfs/dev-support/checkstyle-noframes-sorted.xsl /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDecommission.java /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data120bytes /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data1k /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/editsStored /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/hadoop-dfs-dir.txt /hadoop/common/trunk/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs/src/test/resources/fsimageV18 /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-policy.xml /hadoop/common/trunk/hadoop-hdfs/src/main/java/hdfs-default.xml /hadoop/common/trunk/hadoop-hdfs/src/test/resources/fsimageV19 /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-22-dfs-dir.tgz /hadoop/common/trunk/hadoop-hdfs/ivy/libraries.properties /hadoop/common/trunk/hdfs /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data60bytes /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data60bytes /hadoop/common/trunk/hadoop-hdfs/src/java /hadoop/common/trunk/hadoop-hdfs/src/main/conf/hdfs-site.xml /hadoop/common/trunk/hadoop-hdfs/src/main/native/c++/libhdfs /hadoop/common/trunk/hadoop-hdfs/src/test/resources /hadoop/common/trunk/hadoop-hdfs/src/main/conf/hdfs-site.xml.template /hadoop/common/trunk/hadoop-hdfs/src/main/packages/templates/conf/hdfs-site.xml /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/README.txt /hadoop/common/trunk/hadoop-project/pom.xml /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-test-template.xml /hadoop/common/trunk/hadoop-hdfs/src/main/bin /hadoop/common/trunk/hadoop-hdfs/src/test/resources/log4j.properties /hadoop/common/trunk/hadoop-hdfs/src/test/checkstyle.xml /hadoop/common/trunk/hadoop-project-distro/pom.xml /hadoop/common/trunk/hadoop-hdfs/dev-support/commit-tests /hadoop/common/trunk/hadoop-hdfs/src/main /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineEditsViewer/editsStored.xml /hadoop/common/trunk/hadoop-hdfs/src/docs /hadoop/common/trunk/hadoop-hdfs/dev-support/smoke-tests /hadoop/common/trunk/hadoop-hdfs/src/test/fi-site.xml /hadoop/common/trunk/hadoop-hdfs/src/main/java /hadoop/common/trunk/hadoop-hdfs/src/main/conf /hadoop/common/trunk/hadoop-hdfs/dev-support/findbugsExcludeFile.xml /hadoop/common/trunk/hadoop-hdfs/src/test/resources/editsStored /hadoop/common/trunk/hadoop-hdfs/src/test/checkstyle-noframes-sorted.xsl /hadoop/common/trunk/hadoop-hdfs/src/test/smoke-tests /hadoop/common/trunk/hadoop-hdfs/src/c++ /hadoop/common/trunk/hadoop-hdfs/src/test/findbugsExcludeFile.xml /hadoop/common/trunk/hadoop-hdfs/ivy/hadoop-hdfs-instrumented-test-template.xml /hadoop/common/trunk/hadoop-hdfs/.eclipse.templates/.launches/AllTests.launch /hadoop/common/trunk/hadoop-project-distro /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data1k /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data120bytes /hadoop/common/trunk/hadoop-hdfs/src/test/resources/fi-site.xml /hadoop/common/trunk/hadoop-hdfs/src/test/resources/testHDFSConf.xml /hadoop/common/trunk/hadoop-hdfs/lib/jdiff /hadoop/common/trunk/hadoop-hdfs/src/main/docs /hadoop/common/trunk/hadoop-project-distro/README.txt /hadoop/common/trunk/hadoop-hdfs/src/test/resources/hadoop-dfs-dir.txt /hadoop/common/trunk/hadoop-hdfs/conf /hadoop/common/trunk/hadoop-hdfs/src/test/java/org /hadoop/common/trunk/hadoop-hdfs/src/test/krb5.conf /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/testHDFSConf.xml /hadoop/common/trunk/hadoop-hdfs/src/test/resources/data30bytes /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/fsimageV18 /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/fsimageV19 /hadoop/common/trunk/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/proto/HdfsProtos.java /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/cli/clitest_data/data30bytes /hadoop/common/trunk/hadoop-hdfs/src/test/empty-file /hadoop/common/trunk/hadoop-hdfs/src/main/webapps/proto-secondary-web.xml /hadoop/common/trunk/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/hadoop-14-dfs-dir.tgz /hadoop/common/trunk/hadoop-hdfs/src/main/webapps/proto-datanode-web.xml /hadoop/common/trunk/hadoop-hdfs/src/main/native/c++ /hadoop/common/trunk/hadoop-hdfs/dev-support/jdiff /hadoop/common/trunk/hadoop-hdfs/pom.xml /hadoop/common/trunk/hadoop-hdfs/.gitignore /hadoop/common/trunk/hadoop-hdfs/src/main/resources/hdfs-default.xml /hadoop/common/trunk/hadoop-hdfs/dev-support/test-patch.properties /hadoop/common/trunk/hadoop-hdfs/src/packages /hadoop/common/trunk/hadoop-hdfs/src/test/commit-tests /hadoop/common/trunk/hadoop-hdfs/src/test/resources/editsStored.xml /hadoop/common/trunk/hadoop-hdfs/dev-support/all-tests /hadoop/common/trunk/hadoop-hdfs/src/main/webapps /hadoop/common/trunk/hadoop-hdfs/src/test/resources/krb5.conf
          Hide
          Eric Yang added a comment -

          +1

          (ignoring unit test failure in HADOOP-7264)

          Show
          Eric Yang added a comment - +1 (ignoring unit test failure in HADOOP-7264 )
          Hide
          Alejandro Abdelnur added a comment -

          version 'AJ' Rebased to trunk post MR279. It depends on HADOOP-7264.

          Show
          Alejandro Abdelnur added a comment - version 'AJ' Rebased to trunk post MR279. It depends on HADOOP-7264 .
          Hide
          Alejandro Abdelnur added a comment -

          Rebasing to current trunk

          Show
          Alejandro Abdelnur added a comment - Rebasing to current trunk
          Hide
          Tom White added a comment -

          +1

          I tried various mvn goals, ran test-patch.sh on an example patch, and built MR against Common and HDFS jars built by Maven.

          So I think this is ready to go in - we could do this today (Friday), or wait until Monday if folks think that is better. In terms of migration, I plan to write a patch migration script (like the one in HADOOP-7500 for Common), we'll also need switch over the Jenkins jobs to use Maven (just like Giri and I did for Common).

          Show
          Tom White added a comment - +1 I tried various mvn goals, ran test-patch.sh on an example patch, and built MR against Common and HDFS jars built by Maven. So I think this is ready to go in - we could do this today (Friday), or wait until Monday if folks think that is better. In terms of migration, I plan to write a patch migration script (like the one in HADOOP-7500 for Common), we'll also need switch over the Jenkins jobs to use Maven (just like Giri and I did for Common).
          Hide
          Alejandro Abdelnur added a comment -

          'AH' version of script/patch.

          Fixes issue with some testcases (ie TestHDFSServerPorts) where the test data directory is resolved to '/test/..' instead inside the build directory.

          The reason is that the 'hadoop.tmp.dir' sys prop was not being set up for testcases.

          Show
          Alejandro Abdelnur added a comment - 'AH' version of script/patch. Fixes issue with some testcases (ie TestHDFSServerPorts) where the test data directory is resolved to '/test/..' instead inside the build directory. The reason is that the 'hadoop.tmp.dir' sys prop was not being set up for testcases.
          Hide
          Aaron T. Myers added a comment -

          Thanks a lot for fixing those issues, Alejandro. I re-ran all the tests I ran last night and everything worked great.

          Additionally, I tried out test-patch.sh, which seemed to work flawlessly.

          +1, patch looks good to me.

          Show
          Aaron T. Myers added a comment - Thanks a lot for fixing those issues, Alejandro. I re-ran all the tests I ran last night and everything worked great. Additionally, I tried out test-patch.sh, which seemed to work flawlessly. +1, patch looks good to me.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490146/HDFS-2096-AG.patch
          against trunk revision 1156490.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 68 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1088//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490146/HDFS-2096-AG.patch against trunk revision 1156490. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 68 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1088//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          'AG' script/patch, changes for test-patch & smart-apply-patch script to work with Mavenized HDFS

          Show
          Alejandro Abdelnur added a comment - 'AG' script/patch, changes for test-patch & smart-apply-patch script to work with Mavenized HDFS
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490143/HDFS-2096-AF.patch
          against trunk revision 1156490.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 69 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1087//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490143/HDFS-2096-AF.patch against trunk revision 1156490. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 69 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1087//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          ATM thanks for your feedback/findings.

          'AF' version of script/patch fixes the issues you run into.

          When testing, make sure to do a 'mvn install -DskipTests' from trunk/ before running mvn from trunk/hadoop-hdfs/ as there are changes in the other modules as well.

          Show
          Alejandro Abdelnur added a comment - ATM thanks for your feedback/findings. 'AF' version of script/patch fixes the issues you run into. When testing, make sure to do a 'mvn install -DskipTests' from trunk/ before running mvn from trunk/hadoop-hdfs/ as there are changes in the other modules as well.
          Hide
          Aaron T. Myers added a comment -

          Hey Alejandro, I gave this patch a try and almost everything looked great.

          I tried the following with great success:

          1. Importing the projects into Eclipse. Worked flawlessly except for the known issue with generated sources. (BTW, the instructions on the wiki are slightly off. I had to add the directory "target/generated-src/test/java", not "target/generated-test-sources/java" as are in the instructions.)
          2. Clean/compile the project from trunk/
          3. Clean/compile the project from trunk/hdfs
          4. Run a single unit test
          5. Run the full suite of unit tests
          6. Building a plain TAR
          7. Running findbugs

          The following, however, did not work:

          1. Though the RAT check runs just fine, HDFS fails because of "Too many unapproved licenses: 57". I think a few more directories need to be excluded.
          2. Running checkstyle. I ran `mvn compile checkstyle:checkstyle' as instrucuted, which failed with the following error:
            [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.6:checkstyle (default-cli) on project hadoop-hdfs: An error has occurred in Checkstyle report generation. Failed during checkstyle configuration: cannot initialize module PackageHtml - Unable to instantiate PackageHtml: Unable to instantiate PackageHtmlCheck -> [Help 1]
            


          3. Building a TAR with the native libs. I ran the following from trunk/hadoop-hdfs: mvn -DskipTests -Pnative package. This failed with the following error:
          [INFO] --- make-maven-plugin:1.0-beta-1:test (test) @ hadoop-hdfs ---
          [INFO] make: *** No rule to make target `check'.  Stop.
          

          Somewhat curiously, simply compiling the native libs (`mvn -Pnative compile') appears to work.

          Show
          Aaron T. Myers added a comment - Hey Alejandro, I gave this patch a try and almost everything looked great. I tried the following with great success: Importing the projects into Eclipse. Worked flawlessly except for the known issue with generated sources. (BTW, the instructions on the wiki are slightly off. I had to add the directory " target/generated-src/test/java ", not " target/generated-test-sources/java " as are in the instructions.) Clean/compile the project from trunk/ Clean/compile the project from trunk/hdfs Run a single unit test Run the full suite of unit tests Building a plain TAR Running findbugs The following, however, did not work: Though the RAT check runs just fine, HDFS fails because of "Too many unapproved licenses: 57". I think a few more directories need to be excluded. Running checkstyle. I ran `mvn compile checkstyle:checkstyle' as instrucuted, which failed with the following error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.6:checkstyle (default-cli) on project hadoop-hdfs: An error has occurred in Checkstyle report generation. Failed during checkstyle configuration: cannot initialize module PackageHtml - Unable to instantiate PackageHtml: Unable to instantiate PackageHtmlCheck -> [Help 1] Building a TAR with the native libs. I ran the following from trunk/hadoop-hdfs: mvn -DskipTests -Pnative package . This failed with the following error: [INFO] --- make-maven-plugin:1.0-beta-1:test (test) @ hadoop-hdfs --- [INFO] make: *** No rule to make target `check'. Stop. Somewhat curiously, simply compiling the native libs ( `mvn -Pnative compile' ) appears to work.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490016/HDFS-2096-AE.patch
          against trunk revision 1155998.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 59 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1073//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490016/HDFS-2096-AE.patch against trunk revision 1155998. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 59 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1073//console This message is automatically generated.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12490016/HDFS-2096-AE.patch
          against trunk revision 1155998.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 59 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1068//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12490016/HDFS-2096-AE.patch against trunk revision 1155998. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 59 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1068//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          'AE' version of script/patch, work on current trunk HEAD (revision 1156215)

          Show
          Alejandro Abdelnur added a comment - 'AE' version of script/patch, work on current trunk HEAD (revision 1156215)
          Hide
          Tom White added a comment -

          HADOOP-7533 fixes test-patch to support a Mavenized HDFS.

          Show
          Tom White added a comment - HADOOP-7533 fixes test-patch to support a Mavenized HDFS.
          Hide
          Eli Collins added a comment -

          Yea, let's move forward with this jira and fix HDFS-2224 post mavenization.

          Show
          Eli Collins added a comment - Yea, let's move forward with this jira and fix HDFS-2224 post mavenization.
          Hide
          Tom White added a comment -

          Does this include changes for running test-patch?

          Show
          Tom White added a comment - Does this include changes for running test-patch?
          Hide
          Alejandro Abdelnur added a comment -

          script/patch 'AD'

          The only thing missing in the Maven build is wiring the native testcases.

          Which (the native testcases) have been broken in trunk for a while, HDFS-2224.

          Would it be OK to move forward with HDFS-2096 and then fix HDFS-2224 on top of Mavenized HDFS? (more than fix is implement the native test invocation).

          What we also have to do is to make the test-patch script to work on commons or hdfs to run just the testcases of either commons or hdfs.

          Show
          Alejandro Abdelnur added a comment - script/patch 'AD' The only thing missing in the Maven build is wiring the native testcases. Which (the native testcases) have been broken in trunk for a while, HDFS-2224 . Would it be OK to move forward with HDFS-2096 and then fix HDFS-2224 on top of Mavenized HDFS? (more than fix is implement the native test invocation). What we also have to do is to make the test-patch script to work on commons or hdfs to run just the testcases of either commons or hdfs.
          Hide
          Alejandro Abdelnur added a comment -

          script/patch version 'AC'.

          It is fully functional and the packaging is complete as well (it is shared with hadoop-common).

          The only missing part are the native testcases.

          It introduces a new Maven module, hadoop-project-distro. This module is the parent of hadoop-common & hadoop-distro and all common build instructions are in that POM (forrest, findbugs, checkstyle and TAR packaging).

          It also adds Apache Maven2 SNAPSHOTs repo to the hadoop-project POM. This allows to build from any module level using published snapshot versions for the modules that are not being built.

          Show
          Alejandro Abdelnur added a comment - script/patch version 'AC'. It is fully functional and the packaging is complete as well (it is shared with hadoop-common). The only missing part are the native testcases. It introduces a new Maven module, hadoop-project-distro. This module is the parent of hadoop-common & hadoop-distro and all common build instructions are in that POM (forrest, findbugs, checkstyle and TAR packaging). It also adds Apache Maven2 SNAPSHOTs repo to the hadoop-project POM. This allows to build from any module level using published snapshot versions for the modules that are not being built.
          Hide
          Alejandro Abdelnur added a comment -

          script/patch set 'AB'

          All testcases passing.

          Missing

          • wiring test-patch script for HDFS.
          • the AOP tests (another JIRA)
          • contrib (another JIRA)
          Show
          Alejandro Abdelnur added a comment - script/patch set 'AB' All testcases passing. Missing wiring test-patch script for HDFS. the AOP tests (another JIRA) contrib (another JIRA)
          Hide
          Alejandro Abdelnur added a comment -

          Preview (still a few things missing)

          Patch 'AA'

          Patch instructions (Same as with HADOOP-6671):

          • run the script on trunk/
          • apply the patch on trunk/

          Building instructions are the same as HADOOP-6671 (explained in trunk/BUILDING.txt).

          • It build hdfs native.
          • It runs the testcases (there are some failures I'm looking into) (see at the end of this comment).
          • It creates docs & creates TARs with both legacy tar and new binary layouts.
          • System/unit tests are not wired.
          • Native tests are not wired.

          Of course, it requires HADOOP-6671.

          The build between common and hdfs is identical and it can be factored out (I'll do that as part of follow up JIRA onces things are stable).

          I need some help for the following:

          • Understanding how/when system/unit test are run.
          • Understanding how native tests are setup/run.

          For the latter I'd like to break it into another JIRA as it seems not that trivial and I'll need help.

          Current testcases failures:

          Running org.apache.hadoop.hdfs.TestHDFSServerPorts
          Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 15.483 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.blockmanagement.TestHost2NodesMap
          Tests run: 4, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.061 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
          Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.792 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint
          Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 138.615 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade
          Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.247 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks
          Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 71.016 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream
          Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.645 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage
          Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.965 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.TestDatanodeBlockScanner
          Tests run: 5, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 38.767 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.TestDecommission
          Tests run: 7, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 151.665 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.namenode.TestStartup
          Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 28.425 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
          Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.662 sec <<< FAILURE!
          Running org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark
          Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.519 sec <<< FAILURE!
          Running org.apache.hadoop.cli.TestHDFSCLI
          Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 0.169 sec <<< FAILURE!
          
          Show
          Alejandro Abdelnur added a comment - Preview (still a few things missing) Patch 'AA' Patch instructions (Same as with HADOOP-6671 ): run the script on trunk/ apply the patch on trunk/ Building instructions are the same as HADOOP-6671 (explained in trunk/BUILDING.txt). It build hdfs native. It runs the testcases (there are some failures I'm looking into) (see at the end of this comment). It creates docs & creates TARs with both legacy tar and new binary layouts. System/unit tests are not wired. Native tests are not wired. Of course, it requires HADOOP-6671 . The build between common and hdfs is identical and it can be factored out (I'll do that as part of follow up JIRA onces things are stable). I need some help for the following: Understanding how/when system/unit test are run. Understanding how native tests are setup/run. For the latter I'd like to break it into another JIRA as it seems not that trivial and I'll need help. Current testcases failures: Running org.apache.hadoop.hdfs.TestHDFSServerPorts Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 15.483 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.blockmanagement.TestHost2NodesMap Tests run: 4, Failures: 3, Errors: 0, Skipped: 0, Time elapsed: 0.061 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.blockmanagement.TestReplicationPolicy Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.792 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.namenode.TestCheckpoint Tests run: 6, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 138.615 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.common.TestDistributedUpgrade Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.247 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.namenode.TestBlocksWithNotEnoughRacks Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 71.016 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.namenode.TestEditLogFileOutputStream Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 7.645 sec <<< FAILURE! Running org.apache.hadoop.hdfs.TestDFSUpgradeFromImage Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 0.965 sec <<< FAILURE! Running org.apache.hadoop.hdfs.TestDatanodeBlockScanner Tests run: 5, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 38.767 sec <<< FAILURE! Running org.apache.hadoop.hdfs.TestDecommission Tests run: 7, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 151.665 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.namenode.TestStartup Tests run: 6, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 28.425 sec <<< FAILURE! Running org.apache.hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer Tests run: 2, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 7.662 sec <<< FAILURE! Running org.apache.hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.519 sec <<< FAILURE! Running org.apache.hadoop.cli.TestHDFSCLI Tests run: 2, Failures: 1, Errors: 1, Skipped: 0, Time elapsed: 0.169 sec <<< FAILURE!

            People

            • Assignee:
              Alejandro Abdelnur
              Reporter:
              Alejandro Abdelnur
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development