Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9282

Make data directory count and storage raw capacity related tests FsDataset-agnostic

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Target Version/s:

      Description

      DFSMiniCluster and several tests have hard coded assumption of the underlying storage having 2 data directories (volumes). As HDFS-9188 pointed out, with new FsDataset implementations, these hard coded assumption about number of data directories and raw capacities of storage may change as well.

      We need to extend FsDatasetTestUtils to provide:

      • Number of data directories of underlying storage per DataNode
      • Raw storage capacity of underlying storage per DataNode.
      • Have MiniDFSCluster automatically pick up the correct values.
      1. HDFS-9282.001.patch
        8 kB
        Tony Wu
      2. HDFS-9282.002.patch
        9 kB
        Tony Wu
      3. HDFS-9282.003.patch
        9 kB
        Tony Wu
      4. HDFS-9282.004.patch
        9 kB
        Tony Wu

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #575 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/575/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Hadoop-Hdfs-trunk-Java8 #575 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/575/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2573 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2573/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2573 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2573/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2513 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2513/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2513 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2513/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #643 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/643/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #643 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/643/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #632 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/632/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #632 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/632/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1366 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1366/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1366 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1366/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8762 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8762/)
          HDFS-9282. Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8762 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8762/ ) HDFS-9282 . Make data directory count and storage raw capacity related (lei: rev 46e78a7a1694407d176218c20cb7438ab1335490) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNamenodeCapacityReport.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMultipleRegistrations.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          +1. Thanks Tony Wu

          Show
          eddyxu Lei (Eddy) Xu added a comment - +1. Thanks Tony Wu
          Hide
          twu Tony Wu added a comment -

          The failed test {[TestDeleteRace}} is not relevant to the change. Also manually verified the test passes on a Linux machine.

          Show
          twu Tony Wu added a comment - The failed test {[TestDeleteRace}} is not relevant to the change. Also manually verified the test passes on a Linux machine.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 8s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 3m 8s trunk passed
          +1 compile 0m 30s trunk passed with JDK v1.8.0_60
          +1 compile 0m 31s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 14s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs
          +1 javadoc 1m 6s trunk passed with JDK v1.8.0_60
          +1 javadoc 1m 48s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 38s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.8.0_60
          +1 javac 0m 31s the patch passed
          +1 compile 0m 31s the patch passed with JDK v1.7.0_79
          +1 javac 0m 31s the patch passed
          +1 checkstyle 0m 14s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 3s the patch passed
          +1 javadoc 1m 7s the patch passed with JDK v1.8.0_60
          +1 javadoc 1m 46s the patch passed with JDK v1.7.0_79
          +1 unit 50m 17s hadoop-hdfs in the patch passed with JDK v1.8.0_60.
          -1 unit 50m 38s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 19s Patch generated 56 ASF License warnings.
          120m 18s



          Reason Tests
          JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.namenode.TestDeleteRace



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770693/HDFS-9282.004.patch
          JIRA Issue HDFS-9282
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 2d1774b7fdc4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-e8bd3ad/precommit/personality/hadoop.sh
          git revision trunk / 5667129
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13388/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13388/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 3m 8s trunk passed +1 compile 0m 30s trunk passed with JDK v1.8.0_60 +1 compile 0m 31s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 14s trunk passed +1 mvneclipse 0m 14s trunk passed -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs +1 javadoc 1m 6s trunk passed with JDK v1.8.0_60 +1 javadoc 1m 48s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 38s the patch passed +1 compile 0m 31s the patch passed with JDK v1.8.0_60 +1 javac 0m 31s the patch passed +1 compile 0m 31s the patch passed with JDK v1.7.0_79 +1 javac 0m 31s the patch passed +1 checkstyle 0m 14s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 3s the patch passed +1 javadoc 1m 7s the patch passed with JDK v1.8.0_60 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_79 +1 unit 50m 17s hadoop-hdfs in the patch passed with JDK v1.8.0_60. -1 unit 50m 38s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 19s Patch generated 56 ASF License warnings. 120m 18s Reason Tests JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.namenode.TestDeleteRace Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770693/HDFS-9282.004.patch JIRA Issue HDFS-9282 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 2d1774b7fdc4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-e8bd3ad/precommit/personality/hadoop.sh git revision trunk / 5667129 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13388/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13388/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13388/console This message was automatically generated.
          Hide
          twu Tony Wu added a comment -

          In v4 patch:

          • Addressed Lei (Eddy) Xu's review comments (use try-with-resources to avoid leaking resources).
          Show
          twu Tony Wu added a comment - In v4 patch: Addressed Lei (Eddy) Xu 's review comments (use try-with-resources to avoid leaking resources).
          Hide
          twu Tony Wu added a comment -

          Thanks Lei (Eddy) Xu for catching this. I have incorporated the comments in the latest patch.

          Show
          twu Tony Wu added a comment - Thanks Lei (Eddy) Xu for catching this. I have incorporated the comments in the latest patch.
          Hide
          eddyxu Lei (Eddy) Xu added a comment -
           Preconditions.checkState(dataset.getFsVolumeReferences().size() != 0);
           FsVolumeReferences volRefs = dataset.getFsVolumeReferences();
          try {
          ...
          } finally {
            volRefs.close();
          }
          

          The references are obtained twice but only one has been released. Also you can use try-with-resource to simplify the code.

          The rest LGTM. will +1 once the above comment be addressed.

          Show
          eddyxu Lei (Eddy) Xu added a comment - Preconditions.checkState(dataset.getFsVolumeReferences().size() != 0); FsVolumeReferences volRefs = dataset.getFsVolumeReferences(); try { ... } finally { volRefs.close(); } The references are obtained twice but only one has been released. Also you can use try-with-resource to simplify the code. The rest LGTM. will +1 once the above comment be addressed.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 9s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          +1 mvninstall 3m 13s trunk passed
          +1 compile 0m 40s trunk passed with JDK v1.8.0_60
          +1 compile 0m 37s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 18s trunk passed
          +1 mvneclipse 0m 15s trunk passed
          -1 findbugs 2m 11s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs
          +1 javadoc 1m 19s trunk passed with JDK v1.8.0_60
          +1 javadoc 2m 9s trunk passed with JDK v1.7.0_79
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 40s the patch passed with JDK v1.8.0_60
          +1 javac 0m 40s the patch passed
          +1 compile 0m 36s the patch passed with JDK v1.7.0_79
          +1 javac 0m 36s the patch passed
          +1 checkstyle 0m 18s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 22s the patch passed
          +1 javadoc 1m 22s the patch passed with JDK v1.8.0_60
          +1 javadoc 2m 12s the patch passed with JDK v1.7.0_79
          -1 unit 64m 44s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
          -1 unit 60m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          -1 asflicense 0m 22s Patch generated 56 ASF License warnings.
          147m 38s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.hdfs.TestBlockReaderLocal
            hadoop.hdfs.TestReadStripedFileWithDecoding
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.namenode.TestFSEditLogLoader
          JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.TestDFSInotifyEventInputStream
            hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
            hadoop.hdfs.tools.TestGetGroups



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-30
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769658/HDFS-9282.003.patch
          JIRA Issue HDFS-9282
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile
          uname Linux 3b3c3d9350f4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh
          git revision trunk / 7412ff4
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13288/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Max memory used 227MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13288/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 9s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. +1 mvninstall 3m 13s trunk passed +1 compile 0m 40s trunk passed with JDK v1.8.0_60 +1 compile 0m 37s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 18s trunk passed +1 mvneclipse 0m 15s trunk passed -1 findbugs 2m 11s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs +1 javadoc 1m 19s trunk passed with JDK v1.8.0_60 +1 javadoc 2m 9s trunk passed with JDK v1.7.0_79 +1 mvninstall 0m 45s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_60 +1 javac 0m 40s the patch passed +1 compile 0m 36s the patch passed with JDK v1.7.0_79 +1 javac 0m 36s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 22s the patch passed +1 javadoc 1m 22s the patch passed with JDK v1.8.0_60 +1 javadoc 2m 12s the patch passed with JDK v1.7.0_79 -1 unit 64m 44s hadoop-hdfs in the patch failed with JDK v1.8.0_60. -1 unit 60m 14s hadoop-hdfs in the patch failed with JDK v1.7.0_79. -1 asflicense 0m 22s Patch generated 56 ASF License warnings. 147m 38s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.hdfs.TestBlockReaderLocal   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.namenode.TestFSEditLogLoader JDK v1.7.0_79 Failed junit tests hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.TestDFSInotifyEventInputStream   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover   hadoop.hdfs.tools.TestGetGroups Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-30 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12769658/HDFS-9282.003.patch JIRA Issue HDFS-9282 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile uname Linux 3b3c3d9350f4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-e77b1ce/precommit/personality/hadoop.sh git revision trunk / 7412ff4 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13288/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13288/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Max memory used 227MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13288/console This message was automatically generated.
          Hide
          twu Tony Wu added a comment -

          In v3 patch:

          • Rebased patch on latest trunk.
          Show
          twu Tony Wu added a comment - In v3 patch: Rebased patch on latest trunk.
          Hide
          twu Tony Wu added a comment -

          Manually verified hadoop.hdfs.server.datanode.TestDirectoryScanner runs without error. The patch does not change anything related to this test.

          Show
          twu Tony Wu added a comment - Manually verified hadoop.hdfs.server.datanode.TestDirectoryScanner runs without error. The patch does not change anything related to this test.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 8m 2s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 6 new or modified test files.
          +1 javac 8m 1s There were no new javac warning messages.
          +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 25s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse.
          +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 1m 14s Pre-build of native portion
          -1 hdfs tests 50m 11s Tests failed in hadoop-hdfs.
              73m 47s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768754/HDFS-9282.002.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 123b3db
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13192/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13192/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13192/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13192/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 8m 2s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 6 new or modified test files. +1 javac 8m 1s There were no new javac warning messages. +1 release audit 0m 22s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 25s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 31s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 14s Pre-build of native portion -1 hdfs tests 50m 11s Tests failed in hadoop-hdfs.     73m 47s   Reason Tests Failed unit tests hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768754/HDFS-9282.002.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 123b3db Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13192/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13192/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13192/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13192/console This message was automatically generated.
          Hide
          twu Tony Wu added a comment -

          Addressed Lei (Eddy) Xu's review comments.

          Show
          twu Tony Wu added a comment - Addressed Lei (Eddy) Xu 's review comments.
          Hide
          twu Tony Wu added a comment -

          Hi Lei (Eddy) Xu,

          Thank you very much for the detailed review. I have addressed all of your comments in the new patch. Please take a look.

          Regarding your comment:

          • FsDatasetTestUtils#getNumOfDataDirs() should be renamed as getDefaultNumOfDataDirs(). I was thinking the method in TestUtils may not always return the default value. Instead it may choose to return some calculated value. But this is probably over-designing the API and I have changed the name as you suggested.

          Thanks,
          Tony

          Show
          twu Tony Wu added a comment - Hi Lei (Eddy) Xu , Thank you very much for the detailed review. I have addressed all of your comments in the new patch. Please take a look. Regarding your comment: FsDatasetTestUtils#getNumOfDataDirs() should be renamed as getDefaultNumOfDataDirs() . I was thinking the method in TestUtils may not always return the default value. Instead it may choose to return some calculated value. But this is probably over-designing the API and I have changed the name as you suggested. Thanks, Tony
          Hide
          eddyxu Lei (Eddy) Xu added a comment -

          Thanks a lot for the patch, Tony Wu

          • Could you fix the whitespace warning, which is related.
             /**
               * Get the default number of data directories for underlying storage per
               * DataNode. Used by MiniDFSCluster to initialize cluster.
          

          We can remove Used by MniDFSCluster....

          • FsDatasetTestUtils#getNumOfDataDirs() should also be named as getDefaultNumOfDataDirs() to make it consistent with the Factory.
          • Lets write the following code in separated lines.
            public int getNumOfDataDirs() { return this.defaultNumOfDataDirs; }
            
          • public static final int defaultNumOfDataDirs = 2; should be DEFAULT_NUM_OF_DATA_DIRS, and could you add some comments for it?
          • The FsVolumeReferences from the following code should be closed to release reference counts after usage.
            DF df = new DF(new File(
                    dataset.getFsVolumeReferences().get(0).getBasePath()),
                    dataset.datanode.getConf());
            

          Also for assert (dataset.getFsVolumeReferences().size() != 0);. Btw, can we use Preconditions.checkState to do the checks here?

          Show
          eddyxu Lei (Eddy) Xu added a comment - Thanks a lot for the patch, Tony Wu Could you fix the whitespace warning, which is related. /** * Get the default number of data directories for underlying storage per * DataNode. Used by MiniDFSCluster to initialize cluster. We can remove Used by MniDFSCluster... . FsDatasetTestUtils#getNumOfDataDirs() should also be named as getDefaultNumOfDataDirs() to make it consistent with the Factory. Lets write the following code in separated lines. public int getNumOfDataDirs() { return this .defaultNumOfDataDirs; } public static final int defaultNumOfDataDirs = 2; should be DEFAULT_NUM_OF_DATA_DIRS , and could you add some comments for it? The FsVolumeReferences from the following code should be closed to release reference counts after usage. DF df = new DF( new File( dataset.getFsVolumeReferences().get(0).getBasePath()), dataset.datanode.getConf()); Also for assert (dataset.getFsVolumeReferences().size() != 0); . Btw, can we use Preconditions.checkState to do the checks here?
          Hide
          twu Tony Wu added a comment -

          Manually reran the failed tests and they both pass without error.
          Inspected the patch and did not find any newly added training spaces.

          Show
          twu Tony Wu added a comment - Manually reran the failed tests and they both pass without error. Inspected the patch and did not find any newly added training spaces.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 7m 58s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 6 new or modified test files.
          +1 javac 7m 53s There were no new javac warning messages.
          +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 1m 25s There were no new checkstyle issues.
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 install 1m 27s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 1m 1s Pre-build of native portion
          -1 hdfs tests 58m 55s Tests failed in hadoop-hdfs.
              82m 4s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.ha.TestHASafeMode
          Timed out tests org.apache.hadoop.hdfs.util.TestByteArrayManager



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767923/HDFS-9282.001.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / 3dadf36
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13120/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13120/artifact/patchprocess/whitespace.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13120/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13120/testReport/
          Java 1.7.0_55
          uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13120/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 7m 58s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 6 new or modified test files. +1 javac 7m 53s There were no new javac warning messages. +1 release audit 0m 20s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 1m 25s There were no new checkstyle issues. -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 install 1m 27s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 28s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 1m 1s Pre-build of native portion -1 hdfs tests 58m 55s Tests failed in hadoop-hdfs.     82m 4s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.ha.TestHASafeMode Timed out tests org.apache.hadoop.hdfs.util.TestByteArrayManager Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767923/HDFS-9282.001.patch Optional Tests javac unit findbugs checkstyle git revision trunk / 3dadf36 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13120/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13120/artifact/patchprocess/whitespace.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13120/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13120/testReport/ Java 1.7.0_55 uname Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13120/console This message was automatically generated.
          Hide
          twu Tony Wu added a comment -

          In this patch:

          • Add getNumOfDataDirs() and getRawCapacity() interfaces.
          • Have MiniDFSCluster automatically pick up the correct number of data directories.
          • Updated a few tests where we have hard coded values.
          Show
          twu Tony Wu added a comment - In this patch: Add getNumOfDataDirs() and getRawCapacity() interfaces. Have MiniDFSCluster automatically pick up the correct number of data directories. Updated a few tests where we have hard coded values.

            People

            • Assignee:
              twu Tony Wu
              Reporter:
              twu Tony Wu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development