Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7228

Add an SSD policy into the default BlockStoragePolicySuite

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Currently in the default BlockStoragePolicySuite, we've defined 4 storage policies: LAZY_PERSIST, HOT, WARM, and COLD. Since we have already defined the SSD storage type, it will be useful to also include a SSD related storage policy in the default suite.

      1. HDFS-7228.003.patch
        19 kB
        Jing Zhao
      2. HDFS-7228.003.patch
        19 kB
        Jing Zhao
      3. HDFS-7228.002.patch
        19 kB
        Jing Zhao
      4. HDFS-7228.001.patch
        16 kB
        Jing Zhao
      5. HDFS-7228.000.patch
        16 kB
        Jing Zhao

        Activity

        Hide
        jingzhao Jing Zhao added a comment -

        BTW, we will allow admins to define new policies in the future, and the corresponding jira is HDFS-7076.

        Show
        jingzhao Jing Zhao added a comment - BTW, we will allow admins to define new policies in the future, and the corresponding jira is HDFS-7076 .
        Hide
        jingzhao Jing Zhao added a comment -

        The patch adds a new storage policy named "SSD":

        new BlockStoragePolicy(id, "SSD",
        new StorageType[]{StorageType.SSD}, new StorageType[]{StorageType.DISK},
        new StorageType[]{StorageType.DISK});
        

        I.e., we have:
        1. For new allocated blocks, the policy tries to place all replicas on SSD
        2. If #2 fails because SSD is unavailable, fall back to DISK
        3. For under-replicated block recovery, if SSD is unavailable, fall back to DISK

        Also use this chance to move the predefined policies' names into HdfsConstants.

        Show
        jingzhao Jing Zhao added a comment - The patch adds a new storage policy named "SSD": new BlockStoragePolicy(id, "SSD" , new StorageType[]{StorageType.SSD}, new StorageType[]{StorageType.DISK}, new StorageType[]{StorageType.DISK}); I.e., we have: 1. For new allocated blocks, the policy tries to place all replicas on SSD 2. If #2 fails because SSD is unavailable, fall back to DISK 3. For under-replicated block recovery, if SSD is unavailable, fall back to DISK Also use this chance to move the predefined policies' names into HdfsConstants.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674229/HDFS-7228.000.patch
        against trunk revision db7f165.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        -1 release audit. The applied patch generated 1 release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8394//testReport/
        Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8394//artifact/patchprocess/patchReleaseAuditProblems.txt
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8394//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674229/HDFS-7228.000.patch against trunk revision db7f165. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. -1 release audit . The applied patch generated 1 release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8394//testReport/ Release audit warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/8394//artifact/patchprocess/patchReleaseAuditProblems.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8394//console This message is automatically generated.
        Hide
        sureshms Suresh Srinivas added a comment - - edited

        Jing Zhao, is this policy placing all the replicas in SSD? Instead of that, should we place only one replica in SSD and the remaining in default storage? This may be better given SSD is more expensive than disk and is not as abundant as disk? Applications can place their computation tasks closer to SSD replica (which is possible given block location now includes storage type).

        Show
        sureshms Suresh Srinivas added a comment - - edited Jing Zhao , is this policy placing all the replicas in SSD? Instead of that, should we place only one replica in SSD and the remaining in default storage? This may be better given SSD is more expensive than disk and is not as abundant as disk? Applications can place their computation tasks closer to SSD replica (which is possible given block location now includes storage type).
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks for the comments, Suresh! So in the new patch I just change the new policy to:

        storageTypes=[SSD, DISK], 
        creationFallbacks=[SSD, DISK],
        replicationFallbacks=[SSD, DISK]
        

        Thus the first replica will be placed in SSD, and the remaining will be on DISK. If the cluster is run out of SSD, then DISK is used for both block allocation or replica recovery. This policy also covers the scenario where DISK is unavailable (the policy falls back to SSD then) although it is usually rare in practice.

        Show
        jingzhao Jing Zhao added a comment - Thanks for the comments, Suresh! So in the new patch I just change the new policy to: storageTypes=[SSD, DISK], creationFallbacks=[SSD, DISK], replicationFallbacks=[SSD, DISK] Thus the first replica will be placed in SSD, and the remaining will be on DISK. If the cluster is run out of SSD, then DISK is used for both block allocation or replica recovery. This policy also covers the scenario where DISK is unavailable (the policy falls back to SSD then) although it is usually rare in practice.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674609/HDFS-7228.001.patch
        against trunk revision 178bc50.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8410//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8410//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674609/HDFS-7228.001.patch against trunk revision 178bc50. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8410//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8410//console This message is automatically generated.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -
        • I think we need both all-SSD and one-SSD policies. All-SSD is useful for high performance applications.
        • Since storage policy is not released yet, let's renumber the policy IDs. Otherwise, the upper IDs become all used.
        • Do you also want to add constants for the IDs?
        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - I think we need both all-SSD and one-SSD policies. All-SSD is useful for high performance applications. Since storage policy is not released yet, let's renumber the policy IDs. Otherwise, the upper IDs become all used. Do you also want to add constants for the IDs?
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks for the review, Nicholas! Update the patch to address your comments.

        Show
        jingzhao Jing Zhao added a comment - Thanks for the review, Nicholas! Update the patch to address your comments.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674658/HDFS-7228.002.patch
        against trunk revision 0e57aa3.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The test build failed in hadoop-hdfs-project/hadoop-hdfs

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8415//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8415//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674658/HDFS-7228.002.patch against trunk revision 0e57aa3. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The test build failed in hadoop-hdfs-project/hadoop-hdfs +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8415//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8415//console This message is automatically generated.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        Since LAZY_PERSIST has a _, I think ALLSSD and ONESSD should also have a _.

        +1 patch looks good other then that.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - Since LAZY_PERSIST has a _, I think ALLSSD and ONESSD should also have a _. +1 patch looks good other then that.
        Hide
        jingzhao Jing Zhao added a comment -

        Thanks again for the review, Nicholas! Update the patch to address comments.

        Show
        jingzhao Jing Zhao added a comment - Thanks again for the review, Nicholas! Update the patch to address comments.
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        +1 the latest patch looks good.

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - +1 the latest patch looks good.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674691/HDFS-7228.003.patch
        against trunk revision 5faaba0.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestBlockStoragePolicy
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8416//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8416//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674691/HDFS-7228.003.patch against trunk revision 5faaba0. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestBlockStoragePolicy org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8416//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8416//console This message is automatically generated.
        Hide
        jingzhao Jing Zhao added a comment -

        Update TestBlockStoragePolicy#testDefaultPolicies to make the policy names consistent.

        Show
        jingzhao Jing Zhao added a comment - Update TestBlockStoragePolicy#testDefaultPolicies to make the policy names consistent.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12674717/HDFS-7228.003.patch
        against trunk revision 5faaba0.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
        org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing

        The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8417//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8417//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12674717/HDFS-7228.003.patch against trunk revision 5faaba0. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication org.apache.hadoop.hdfs.server.namenode.ha.TestDNFencing The following test timeouts occurred in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/8417//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/8417//console This message is automatically generated.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #6259 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6259/)
        HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6259 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6259/ ) HDFS-7228 . Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
        Hide
        jingzhao Jing Zhao added a comment -

        I've committed this to trunk, branch-2, and branch-2.6. Thanks Nicholas and Suresh for review!

        Show
        jingzhao Jing Zhao added a comment - I've committed this to trunk, branch-2, and branch-2.6. Thanks Nicholas and Suresh for review!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #712 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/712/)
        HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #712 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/712/ ) HDFS-7228 . Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #1902 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1902/)
        HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #1902 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1902/ ) HDFS-7228 . Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1927 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1927/)
        HDFS-7228. Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1927 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1927/ ) HDFS-7228 . Add an SSD policy into the default BlockStoragePolicySuite. Contributed by Jing Zhao. (jing9: rev 7dcad84143a9eef059688570cd0f9cf73747f2de) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/mover/TestStorageMover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSOutputStream.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestBlockStoragePolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockStoragePolicySuite.java
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #6311 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6311/)
        HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #6311 (See https://builds.apache.org/job/Hadoop-trunk-Commit/6311/ ) HDFS-7228 . Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Yarn-trunk #721 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/721/)
        HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f)

        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #721 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/721/ ) HDFS-7228 . Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Hdfs-trunk #1910 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1910/)
        HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1910 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1910/ ) HDFS-7228 . Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk #1935 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1935/)
        HDFS-7228. Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1935 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1935/ ) HDFS-7228 . Fix TestDNFencing.testQueueingWithAppend. Contributed by Yongjun Zhang. (jing9: rev 1c8d191117de3d2e035bd728bccfde0f4b81296f) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestDNFencing.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            jingzhao Jing Zhao
            Reporter:
            jingzhao Jing Zhao
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development