Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11545

Propagate DataNode's slow disks info to the NameNode via Heartbeat

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      DataNode detects the outliers (slow disks) among all its disk. This information can be propagated to the NameNode so that the NameNode gets disk information from all Datanodes.

      1. HDFS-11545.001.patch
        44 kB
        Hanisha Koneru
      2. HDFS-11545.002.patch
        49 kB
        Hanisha Koneru
      3. HDFS-11545.003.patch
        48 kB
        Hanisha Koneru
      4. HDFS-11545-branch-2.001.patch
        49 kB
        Hanisha Koneru

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 12 new or modified test files.
        0 mvndep 0m 7s Maven dependency ordering for branch
        +1 mvninstall 12m 51s trunk passed
        +1 compile 1m 26s trunk passed
        +1 checkstyle 0m 51s trunk passed
        +1 mvnsite 1m 31s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 3m 25s trunk passed
        +1 javadoc 1m 3s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 20s the patch passed
        +1 compile 1m 23s the patch passed
        +1 cc 1m 23s the patch passed
        +1 javac 1m 23s the patch passed
        -0 checkstyle 0m 48s hadoop-hdfs-project: The patch generated 56 new + 696 unchanged - 5 fixed = 752 total (was 701)
        +1 mvnsite 1m 21s the patch passed
        +1 mvneclipse 0m 21s the patch passed
        -1 whitespace 0m 0s The patch 75 line(s) with tabs.
        +1 findbugs 3m 26s the patch passed
        +1 javadoc 0m 57s the patch passed
        +1 unit 0m 55s hadoop-hdfs-client in the patch passed.
        +1 unit 66m 54s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        101m 21s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11545
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859379/HDFS-11545.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 7935e853c3ab 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7f8e928
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18761/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/18761/artifact/patchprocess/whitespace-tabs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18761/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18761/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 12 new or modified test files. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 12m 51s trunk passed +1 compile 1m 26s trunk passed +1 checkstyle 0m 51s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 25s trunk passed +1 javadoc 1m 3s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 20s the patch passed +1 compile 1m 23s the patch passed +1 cc 1m 23s the patch passed +1 javac 1m 23s the patch passed -0 checkstyle 0m 48s hadoop-hdfs-project: The patch generated 56 new + 696 unchanged - 5 fixed = 752 total (was 701) +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 21s the patch passed -1 whitespace 0m 0s The patch 75 line(s) with tabs. +1 findbugs 3m 26s the patch passed +1 javadoc 0m 57s the patch passed +1 unit 0m 55s hadoop-hdfs-client in the patch passed. +1 unit 66m 54s hadoop-hdfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 101m 21s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11545 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859379/HDFS-11545.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 7935e853c3ab 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7f8e928 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18761/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/18761/artifact/patchprocess/whitespace-tabs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18761/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18761/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks Hanisha Koneru. The patch looks great. Minor feedback:

        1. The following comment in DatanodeProtocol.proto looks unnecessary since SlowDiskReportProto does not include the DataNodeId.
           * DataNode. (DataNodeId must be transmitted as a string for protocol
           * compability with earlier versions of Hadoop).
          
        2. NUM_DISK_OPS can be initialized to DiskOp.values().length() instead of hard-coding 3.
        3. Extra space in return slowDiskInfoProtos.

        We should add a few unit tests:

        1. Test for PBHelper serialization/deserialization. See TestPBHelper#testSlowPeerInfoPBHelper.
        2. Ensure that the datanode sends out a slow disk report when one is due, and it includes the contents of dn.getDiskMetrics().getDiskOutliersStats().
        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks Hanisha Koneru . The patch looks great. Minor feedback: The following comment in DatanodeProtocol.proto looks unnecessary since SlowDiskReportProto does not include the DataNodeId. * DataNode. (DataNodeId must be transmitted as a string for protocol * compability with earlier versions of Hadoop). NUM_DISK_OPS can be initialized to DiskOp.values().length() instead of hard-coding 3. Extra space in return slowDiskInfoProtos . We should add a few unit tests: Test for PBHelper serialization/deserialization. See TestPBHelper#testSlowPeerInfoPBHelper. Ensure that the datanode sends out a slow disk report when one is due, and it includes the contents of dn.getDiskMetrics().getDiskOutliersStats() .
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thank you Arpit Agarwal for reviewing the patch. I have updated the patch.

        I was thinking we can add an end-to-end test case for the slow disk report along with the the NameNode handling of the report (HDFS-11551).

        Show
        hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal for reviewing the patch. I have updated the patch. I was thinking we can add an end-to-end test case for the slow disk report along with the the NameNode handling of the report ( HDFS-11551 ).
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the v2 patch pending Jenkins.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the v2 patch pending Jenkins.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 15 new or modified test files.
        0 mvndep 0m 25s Maven dependency ordering for branch
        +1 mvninstall 12m 51s trunk passed
        +1 compile 1m 30s trunk passed
        +1 checkstyle 0m 50s trunk passed
        +1 mvnsite 1m 29s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 3m 16s trunk passed
        +1 javadoc 1m 1s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 17s the patch passed
        +1 compile 1m 20s the patch passed
        +1 cc 1m 20s the patch passed
        +1 javac 1m 20s the patch passed
        -0 checkstyle 0m 48s hadoop-hdfs-project: The patch generated 8 new + 758 unchanged - 5 fixed = 766 total (was 763)
        +1 mvnsite 1m 20s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 33s the patch passed
        +1 javadoc 0m 56s the patch passed
        +1 unit 1m 1s hadoop-hdfs-client in the patch passed.
        -1 unit 66m 58s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        101m 33s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11545
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859638/HDFS-11545.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux c9ba2e5cde33 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6c399a8
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18773/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18773/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18773/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18773/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 15 new or modified test files. 0 mvndep 0m 25s Maven dependency ordering for branch +1 mvninstall 12m 51s trunk passed +1 compile 1m 30s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 29s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 16s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 1m 20s the patch passed +1 cc 1m 20s the patch passed +1 javac 1m 20s the patch passed -0 checkstyle 0m 48s hadoop-hdfs-project: The patch generated 8 new + 758 unchanged - 5 fixed = 766 total (was 763) +1 mvnsite 1m 20s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 33s the patch passed +1 javadoc 0m 56s the patch passed +1 unit 1m 1s hadoop-hdfs-client in the patch passed. -1 unit 66m 58s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 101m 33s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11545 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859638/HDFS-11545.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux c9ba2e5cde33 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6c399a8 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18773/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18773/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18773/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18773/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Resolved checkstyle errors in patch v03.

        Show
        hanishakoneru Hanisha Koneru added a comment - Resolved checkstyle errors in patch v03.
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 14 new or modified test files.
        0 mvndep 0m 24s Maven dependency ordering for branch
        +1 mvninstall 14m 23s trunk passed
        +1 compile 1m 26s trunk passed
        +1 checkstyle 0m 50s trunk passed
        +1 mvnsite 1m 24s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 3m 11s trunk passed
        +1 javadoc 1m 0s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 17s the patch passed
        +1 compile 1m 18s the patch passed
        +1 cc 1m 18s the patch passed
        +1 javac 1m 18s the patch passed
        -0 checkstyle 0m 47s hadoop-hdfs-project: The patch generated 5 new + 759 unchanged - 5 fixed = 764 total (was 764)
        +1 mvnsite 1m 21s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 31s the patch passed
        +1 javadoc 0m 55s the patch passed
        +1 unit 1m 0s hadoop-hdfs-client in the patch passed.
        +1 unit 68m 28s hadoop-hdfs in the patch passed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        104m 16s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11545
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859675/HDFS-11545.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 408178eb0098 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b104f3a
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18777/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18777/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18777/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 14 new or modified test files. 0 mvndep 0m 24s Maven dependency ordering for branch +1 mvninstall 14m 23s trunk passed +1 compile 1m 26s trunk passed +1 checkstyle 0m 50s trunk passed +1 mvnsite 1m 24s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 11s trunk passed +1 javadoc 1m 0s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 17s the patch passed +1 compile 1m 18s the patch passed +1 cc 1m 18s the patch passed +1 javac 1m 18s the patch passed -0 checkstyle 0m 47s hadoop-hdfs-project: The patch generated 5 new + 759 unchanged - 5 fixed = 764 total (was 764) +1 mvnsite 1m 21s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 31s the patch passed +1 javadoc 0m 55s the patch passed +1 unit 1m 0s hadoop-hdfs-client in the patch passed. +1 unit 68m 28s hadoop-hdfs in the patch passed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 104m 16s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11545 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859675/HDFS-11545.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 408178eb0098 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b104f3a Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18777/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18777/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18777/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        The elusive Jenkins +1!

        Thanks for fixing the checkstyle issues Hanisha Koneru. I've committed this to trunk. The branch-2 backport hit a few compile issues with type inference.

        Show
        arpitagarwal Arpit Agarwal added a comment - The elusive Jenkins +1! Thanks for fixing the checkstyle issues Hanisha Koneru . I've committed this to trunk. The branch-2 backport hit a few compile issues with type inference.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11432 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11432/)
        HDFS-11545. Propagate DataNode's slow disks info to the NameNode via (arp: rev e7c8da614c37e36fb8081234f4c639d6054f6082)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBpServiceActorScheduler.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SlowDiskReports.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocolPB/TestPBHelper.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMXBean.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowPeerTracker.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeDiskMetrics.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11432 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11432/ ) HDFS-11545 . Propagate DataNode's slow disks info to the NameNode via (arp: rev e7c8da614c37e36fb8081234f4c639d6054f6082) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolServerSideTranslatorPB.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/InternalDataNodeTestUtils.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDatanodeProtocolRetryPolicy.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBpServiceActorScheduler.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/NNThroughputBenchmark.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBPOfferService.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestFsDatasetCache.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeLifeline.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java (add) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SlowDiskReports.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/protocol/DatanodeProtocol.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/protocolPB/TestPBHelper.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestStorageReport.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/proto/DatanodeProtocol.proto (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeMXBean.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/DatanodeProtocolClientSideTranslatorPB.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowPeerTracker.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestNameNodePrunesMissingStorages.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeDiskMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thank you Arpit Agarwal.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal .
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the branch-2 patch. I've committed it after running the affected unit tests locally with JDK 7.

        Thanks Hanisha Koneru.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the branch-2 patch. I've committed it after running the affected unit tests locally with JDK 7. Thanks Hanisha Koneru .

          People

          • Assignee:
            hanishakoneru Hanisha Koneru
            Reporter:
            hanishakoneru Hanisha Koneru
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development