Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Similar to how DataNodes collect peer performance statistics, we can collect disk performance statistics per datanode and detect outliers among them, if any.

      1. HDFS-11461.000.patch
        34 kB
        Hanisha Koneru
      2. HDFS-11461.001.patch
        34 kB
        Hanisha Koneru
      3. HDFS-11461.002.patch
        36 kB
        Hanisha Koneru
      4. HDFS-11461-branch-2.000.patch
        36 kB
        Hanisha Koneru

        Issue Links

          Activity

          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 12m 49s trunk passed
          +1 compile 0m 51s trunk passed
          +1 checkstyle 0m 37s trunk passed
          +1 mvnsite 0m 58s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 45s trunk passed
          +1 javadoc 0m 41s trunk passed
          +1 mvninstall 0m 57s the patch passed
          +1 compile 0m 49s the patch passed
          +1 javac 0m 49s the patch passed
          +1 checkstyle 0m 33s the patch passed
          +1 mvnsite 0m 53s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 2s The patch has no ill-formed XML file.
          +1 findbugs 1m 56s the patch passed
          +1 javadoc 0m 38s the patch passed
          -1 unit 93m 44s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          119m 36s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11461
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854640/HDFS-11461.000.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux bf59d8a92040 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 4a58870
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18443/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18443/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18443/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 12m 49s trunk passed +1 compile 0m 51s trunk passed +1 checkstyle 0m 37s trunk passed +1 mvnsite 0m 58s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 41s trunk passed +1 mvninstall 0m 57s the patch passed +1 compile 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. +1 findbugs 1m 56s the patch passed +1 javadoc 0m 38s the patch passed -1 unit 93m 44s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 119m 36s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11461 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12854640/HDFS-11461.000.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux bf59d8a92040 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 4a58870 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18443/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18443/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18443/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          aw Allen Wittenauer added a comment -

          detect outliers among them

          How do you determine if the disk traffic is actually from HDFS or from something else? Isn't this data better collected at the system level rather than from Hadoop?

          Show
          aw Allen Wittenauer added a comment - detect outliers among them How do you determine if the disk traffic is actually from HDFS or from something else? Isn't this data better collected at the system level rather than from Hadoop?
          Hide
          hanishakoneru Hanisha Koneru added a comment -

          The intention here is to catch slow disks, as opposed to overloaded disks. So it would not matter if the disk traffic is actually from HDFS or not. We want to find out the overall latency of disk over a long time interval to avoid false positives due to transient traffic.

          This information will only be exposed via jmx so that admins can get a reference on potentially slow disks. They can then run diagnostics or take whatever action as deemed fit.

          Show
          hanishakoneru Hanisha Koneru added a comment - The intention here is to catch slow disks, as opposed to overloaded disks. So it would not matter if the disk traffic is actually from HDFS or not. We want to find out the overall latency of disk over a long time interval to avoid false positives due to transient traffic. This information will only be exposed via jmx so that admins can get a reference on potentially slow disks. They can then run diagnostics or take whatever action as deemed fit.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          Hanisha Koneru, a few comments:

          1. DataNode#shutdown should use slowDiskDetectionThread.join() instead of sleep.
          2. diskOutliers should maintain the mean read/write/meta latency for each flagged disk.
          3. The low threshold should be higher than 1ms (average seek latency of a 7200 RPM disk is 4ms). Let's conservatively set this to 20ms.
          4. startDiskOutlierDetectionThread should call Thread.currentThread().interrupt() after catching InterruptedException. See https://www.ibm.com/developerworks/library/j-jtp05236/
          5. slowDiskDetectionThread should be a daemon thread.
          6. DataNodePeerMetrics should also use OutlierDetector constructor that accepts minNumResources, and pass 10, to keep the behavior consistent with what we have.
          Show
          arpitagarwal Arpit Agarwal added a comment - - edited Hanisha Koneru , a few comments: DataNode#shutdown should use slowDiskDetectionThread.join() instead of sleep. diskOutliers should maintain the mean read/write/meta latency for each flagged disk. The low threshold should be higher than 1ms (average seek latency of a 7200 RPM disk is 4ms). Let's conservatively set this to 20ms. startDiskOutlierDetectionThread should call Thread.currentThread().interrupt() after catching InterruptedException. See https://www.ibm.com/developerworks/library/j-jtp05236/ slowDiskDetectionThread should be a daemon thread. DataNodePeerMetrics should also use OutlierDetector constructor that accepts minNumResources, and pass 10 , to keep the behavior consistent with what we have.
          Hide
          hanishakoneru Hanisha Koneru added a comment -

          Thank you Arpit Agarwal for reviewing the patch. I have addressed your comments in patch v01.

          Show
          hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal for reviewing the patch. I have addressed your comments in patch v01.
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          Thanks Hanisha Koneru. The patch looks good. A few more comments:

          1. diskOutliersStats should be updated atomically. Else if someone reads the map after the clear call but before the for loop completes they will see partial results. You can allocate a new map in detectAndUpdateDiskOutliers and replace the reference which will be atomic.
          2. We can move startDiskOutlierDetectionThread, detectAndUpdateDiskOutliers and diskOutliersStats to a new package-private class. Also the shutdown logic can be moved to a method of the new class. e.g. shutdownAndWait. Sorry I should have suggested this before.
          3. diskOutliersStats can be defined as a Map<String, Map<OpType, Double>>, so that it's explicit which values are read/write/meta.

          Also once the logic is isolated to a new class, see if you can write a unit test (mock-based may be easiest). Ok if you want to file a separate Jira to do that.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited Thanks Hanisha Koneru . The patch looks good. A few more comments: diskOutliersStats should be updated atomically. Else if someone reads the map after the clear call but before the for loop completes they will see partial results. You can allocate a new map in detectAndUpdateDiskOutliers and replace the reference which will be atomic. We can move startDiskOutlierDetectionThread, detectAndUpdateDiskOutliers and diskOutliersStats to a new package-private class. Also the shutdown logic can be moved to a method of the new class. e.g. shutdownAndWait. Sorry I should have suggested this before. diskOutliersStats can be defined as a Map<String, Map<OpType, Double>>, so that it's explicit which values are read/write/meta. Also once the logic is isolated to a new class, see if you can write a unit test (mock-based may be easiest). Ok if you want to file a separate Jira to do that.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 12m 44s trunk passed
          +1 compile 0m 46s trunk passed
          +1 checkstyle 0m 34s trunk passed
          +1 mvnsite 0m 50s trunk passed
          +1 mvneclipse 0m 13s trunk passed
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 0m 40s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 43s the patch passed
          +1 javac 0m 43s the patch passed
          +1 checkstyle 0m 32s the patch passed
          +1 mvnsite 0m 48s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 63m 37s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          88m 18s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestMaintenanceState



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11461
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855197/HDFS-11461.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux 7a67823c3375 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / e0bb867
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18470/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18470/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18470/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 12m 44s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 34s trunk passed +1 mvnsite 0m 50s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed +1 checkstyle 0m 32s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 63m 37s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 88m 18s Reason Tests Failed junit tests hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11461 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855197/HDFS-11461.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux 7a67823c3375 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e0bb867 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18470/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18470/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18470/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          hanishakoneru Hanisha Koneru added a comment - - edited

          Thank you Arpit Agarwal. I have addressed your comments in patch v02.
          I have filed a new Jira for unit test case - HDFS-11475.

          Show
          hanishakoneru Hanisha Koneru added a comment - - edited Thank you Arpit Agarwal . I have addressed your comments in patch v02. I have filed a new Jira for unit test case - HDFS-11475 .
          Hide
          arpitagarwal Arpit Agarwal added a comment - - edited

          +1 for the v3 patch (pending Jenkins). I will hold off committing for a couple of days.

          Show
          arpitagarwal Arpit Agarwal added a comment - - edited +1 for the v3 patch (pending Jenkins). I will hold off committing for a couple of days.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 13m 12s trunk passed
          +1 compile 0m 54s trunk passed
          +1 checkstyle 0m 38s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvneclipse 0m 14s trunk passed
          +1 findbugs 1m 47s trunk passed
          +1 javadoc 0m 42s trunk passed
          +1 mvninstall 0m 50s the patch passed
          +1 compile 0m 48s the patch passed
          +1 javac 0m 48s the patch passed
          +1 checkstyle 0m 33s the patch passed
          +1 mvnsite 0m 58s the patch passed
          +1 mvneclipse 0m 13s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 2m 1s the patch passed
          +1 javadoc 0m 40s the patch passed
          -1 unit 103m 13s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          129m 55s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.TestMaintenanceState



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11461
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855236/HDFS-11461.002.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml
          uname Linux ff872e650466 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 235203d
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/18472/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18472/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18472/console
          Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 13m 12s trunk passed +1 compile 0m 54s trunk passed +1 checkstyle 0m 38s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 42s trunk passed +1 mvninstall 0m 50s the patch passed +1 compile 0m 48s the patch passed +1 javac 0m 48s the patch passed +1 checkstyle 0m 33s the patch passed +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 2m 1s the patch passed +1 javadoc 0m 40s the patch passed -1 unit 103m 13s hadoop-hdfs in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 129m 55s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11461 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12855236/HDFS-11461.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml uname Linux ff872e650466 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 235203d Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18472/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18472/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18472/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          I've committed this to trunk. Thanks for the contribution Hanisha Koneru.

          Show
          arpitagarwal Arpit Agarwal added a comment - I've committed this to trunk. Thanks for the contribution Hanisha Koneru .
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11332 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11332/)
          HDFS-11461. DataNode Disk Outlier Detection. Contributed by Hanisha (arp: rev b3ec531f400dd0a6506dc71233d38ae57b764a43)

          • (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/SlowNodeDetector.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeDiskMetrics.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowPeerTracker.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodePeerMetrics.java
          • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/OutlierDetector.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/metrics/TestDataNodeOutlierDetectionViaMetrics.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/metrics/TestSlowNodeDetector.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11332 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11332/ ) HDFS-11461 . DataNode Disk Outlier Detection. Contributed by Hanisha (arp: rev b3ec531f400dd0a6506dc71233d38ae57b764a43) (delete) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/SlowNodeDetector.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeDiskMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowPeerTracker.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodePeerMetrics.java (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/OutlierDetector.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/metrics/TestDataNodeOutlierDetectionViaMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/metrics/TestSlowNodeDetector.java
          Hide
          hanishakoneru Hanisha Koneru added a comment -

          Thank you Arpit Agarwal for commiting the patch.

          Show
          hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal for commiting the patch.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 for the branch-2 patch. I have committed it.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 for the branch-2 patch. I have committed it.

            People

            • Assignee:
              hanishakoneru Hanisha Koneru
              Reporter:
              hanishakoneru Hanisha Koneru
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development