Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11551

Handle SlowDiskReport from DataNode at the NameNode

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: hdfs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      DataNodes send slow disk reports via heartbeats. Handle these reports at the NameNode to find the topN slow disks.

      1. HDFS-11551.001.patch
        30 kB
        Hanisha Koneru
      2. HDFS-11551.002.patch
        31 kB
        Hanisha Koneru
      3. HDFS-11551.003.patch
        32 kB
        Hanisha Koneru
      4. HDFS-11551.004.patch
        32 kB
        Hanisha Koneru
      5. HDFS-11551.005.patch
        32 kB
        Hanisha Koneru
      6. HDFS-11551.006.patch
        37 kB
        Hanisha Koneru
      7. HDFS-11551.007.patch
        37 kB
        Hanisha Koneru
      8. HDFS-11551.008.patch
        34 kB
        Hanisha Koneru
      9. HDFS-11551.009.patch
        34 kB
        Hanisha Koneru
      10. HDFS-11551.010.patch
        35 kB
        Hanisha Koneru
      11. HDFS-11551-branch-2.001.patch
        35 kB
        Hanisha Koneru
      12. HDFS-11551-branch-2.002.patch
        35 kB
        Hanisha Koneru

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 14m 44s trunk passed
        +1 compile 1m 29s trunk passed
        +1 checkstyle 0m 43s trunk passed
        +1 mvnsite 1m 31s trunk passed
        +1 mvneclipse 0m 28s trunk passed
        +1 findbugs 3m 30s trunk passed
        +1 javadoc 1m 3s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 24s the patch passed
        +1 compile 1m 27s the patch passed
        +1 javac 1m 27s the patch passed
        -0 checkstyle 0m 40s hadoop-hdfs-project: The patch generated 9 new + 44 unchanged - 0 fixed = 53 total (was 44)
        +1 mvnsite 1m 26s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        -1 findbugs 2m 22s hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0)
        +1 javadoc 1m 10s the patch passed
        +1 unit 1m 10s hadoop-hdfs-client in the patch passed.
        -1 unit 75m 8s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        113m 8s



        Reason Tests
        FindBugs module:hadoop-hdfs-project/hadoop-hdfs
          org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.addSlowDiskReport(String, SlowDiskReports) makes inefficient use of keySet iterator instead of entrySet iterator At SlowDiskTracker.java:of keySet iterator instead of entrySet iterator At SlowDiskTracker.java:[line 114]
          org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.getReportsForTesting() makes inefficient use of keySet iterator instead of entrySet iterator At SlowDiskTracker.java:keySet iterator instead of entrySet iterator At SlowDiskTracker.java:[line 263]
          org.apache.hadoop.hdfs.server.datanode.metrics.DataNodeDiskMetrics.detectAndUpdateDiskOutliers(Map, Map, Map) makes inefficient use of keySet iterator instead of entrySet iterator At DataNodeDiskMetrics.java:use of keySet iterator instead of entrySet iterator At DataNodeDiskMetrics.java:[line 133]
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestDistributedFileSystem



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859804/HDFS-11551.001.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 98c6e6f45ad8 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / cc938e9
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18792/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18792/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18792/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18792/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18792/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 14m 44s trunk passed +1 compile 1m 29s trunk passed +1 checkstyle 0m 43s trunk passed +1 mvnsite 1m 31s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 30s trunk passed +1 javadoc 1m 3s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 24s the patch passed +1 compile 1m 27s the patch passed +1 javac 1m 27s the patch passed -0 checkstyle 0m 40s hadoop-hdfs-project: The patch generated 9 new + 44 unchanged - 0 fixed = 53 total (was 44) +1 mvnsite 1m 26s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 2m 22s hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) +1 javadoc 1m 10s the patch passed +1 unit 1m 10s hadoop-hdfs-client in the patch passed. -1 unit 75m 8s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 113m 8s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.addSlowDiskReport(String, SlowDiskReports) makes inefficient use of keySet iterator instead of entrySet iterator At SlowDiskTracker.java:of keySet iterator instead of entrySet iterator At SlowDiskTracker.java: [line 114]   org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.getReportsForTesting() makes inefficient use of keySet iterator instead of entrySet iterator At SlowDiskTracker.java:keySet iterator instead of entrySet iterator At SlowDiskTracker.java: [line 263]   org.apache.hadoop.hdfs.server.datanode.metrics.DataNodeDiskMetrics.detectAndUpdateDiskOutliers(Map, Map, Map) makes inefficient use of keySet iterator instead of entrySet iterator At DataNodeDiskMetrics.java:use of keySet iterator instead of entrySet iterator At DataNodeDiskMetrics.java: [line 133] Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestDistributedFileSystem Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859804/HDFS-11551.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 98c6e6f45ad8 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / cc938e9 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18792/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18792/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/18792/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18792/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18792/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Fixed checkstyle and findbug errors in patch v02.

        Failed junit tests:

        • TestDistributedFileSystem: passes locally
        • TestDataNodeVolumeFailureReporting: fails in a bunch of builds - HDFS-11488
        Show
        hanishakoneru Hanisha Koneru added a comment - Fixed checkstyle and findbug errors in patch v02. Failed junit tests: TestDistributedFileSystem : passes locally TestDataNodeVolumeFailureReporting : fails in a bunch of builds - HDFS-11488
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 27s Maven dependency ordering for branch
        +1 mvninstall 12m 45s trunk passed
        +1 compile 1m 21s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 1m 23s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 6s trunk passed
        +1 javadoc 1m 1s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 16s the patch passed
        +1 compile 1m 20s the patch passed
        +1 javac 1m 20s the patch passed
        +1 checkstyle 0m 37s the patch passed
        +1 mvnsite 1m 18s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 20s the patch passed
        +1 javadoc 0m 53s the patch passed
        +1 unit 0m 53s hadoop-hdfs-client in the patch passed.
        -1 unit 63m 43s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        96m 50s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestMaintenanceState



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859986/HDFS-11551.002.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux e0846fd554b4 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / f462e1f
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18799/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18799/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 12m 45s trunk passed +1 compile 1m 21s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 23s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 6s trunk passed +1 javadoc 1m 1s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 16s the patch passed +1 compile 1m 20s the patch passed +1 javac 1m 20s the patch passed +1 checkstyle 0m 37s the patch passed +1 mvnsite 1m 18s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 20s the patch passed +1 javadoc 0m 53s the patch passed +1 unit 0m 53s hadoop-hdfs-client in the patch passed. -1 unit 63m 43s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 96m 50s Reason Tests Failed junit tests hadoop.hdfs.TestMaintenanceState Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859986/HDFS-11551.002.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e0846fd554b4 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / f462e1f Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18799/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18799/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18799/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for this improvement Hanisha Koneru. The patch looks pretty good.

        One suggestion - we can change the format of the JSON report to have one entry per disk instead of one per DiskOp.

        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for this improvement Hanisha Koneru . The patch looks pretty good. One suggestion - we can change the format of the JSON report to have one entry per disk instead of one per DiskOp.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thank you Arpit Agarwal for reviewing the patch. I have addressed your comments in patch v03.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal for reviewing the patch. I have addressed your comments in patch v03.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 12s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 31s Maven dependency ordering for branch
        +1 mvninstall 13m 45s trunk passed
        +1 compile 1m 24s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 1m 23s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 11s trunk passed
        +1 javadoc 1m 0s trunk passed
        0 mvndep 0m 6s Maven dependency ordering for patch
        +1 mvninstall 1m 16s the patch passed
        +1 compile 1m 19s the patch passed
        +1 javac 1m 19s the patch passed
        -0 checkstyle 0m 38s hadoop-hdfs-project: The patch generated 7 new + 44 unchanged - 0 fixed = 51 total (was 44)
        +1 mvnsite 1m 19s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0)
        +1 javadoc 0m 55s the patch passed
        +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
        -1 unit 68m 36s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        103m 4s



        Reason Tests
        FindBugs module:hadoop-hdfs-project/hadoop-hdfs
          org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.getSlowDiskIDForReport(String, String) invokes inefficient new String(String) constructor At SlowDiskTracker.java:String(String) constructor At SlowDiskTracker.java:[line 114]
          org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency defines equals(SlowDiskTracker$DiskLatency) method and uses Object.equals(Object) At SlowDiskTracker.java:uses Object.equals(Object) At SlowDiskTracker.java:[lines 165-169]
          org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency defines equals and uses Object.hashCode() At SlowDiskTracker.java:Object.hashCode() At SlowDiskTracker.java:[lines 165-169]
        Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMXBean
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860294/HDFS-11551.003.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b339b557db59 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ab759e9
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18823/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18823/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18823/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18823/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18823/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 13m 45s trunk passed +1 compile 1m 24s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 1m 23s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 11s trunk passed +1 javadoc 1m 0s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 16s the patch passed +1 compile 1m 19s the patch passed +1 javac 1m 19s the patch passed -0 checkstyle 0m 38s hadoop-hdfs-project: The patch generated 7 new + 44 unchanged - 0 fixed = 51 total (was 44) +1 mvnsite 1m 19s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. -1 findbugs 1m 53s hadoop-hdfs-project/hadoop-hdfs generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) +1 javadoc 0m 55s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. -1 unit 68m 36s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 103m 4s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.getSlowDiskIDForReport(String, String) invokes inefficient new String(String) constructor At SlowDiskTracker.java:String(String) constructor At SlowDiskTracker.java: [line 114]   org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency defines equals(SlowDiskTracker$DiskLatency) method and uses Object.equals(Object) At SlowDiskTracker.java:uses Object.equals(Object) At SlowDiskTracker.java: [lines 165-169]   org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency defines equals and uses Object.hashCode() At SlowDiskTracker.java:Object.hashCode() At SlowDiskTracker.java: [lines 165-169] Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860294/HDFS-11551.003.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b339b557db59 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ab759e9 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18823/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18823/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/18823/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18823/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18823/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Fixed checkstyle and findbug errors in patch v04.

        Failed junit tests:

        • TestNameNodeMXBean: passes locally
        • TestLazyPersistReplicaRecovery: passes locally
        • TestDataNodeVolumeFailureReporting: fails in a bunch of builds- HDFS-11488
        Show
        hanishakoneru Hanisha Koneru added a comment - Fixed checkstyle and findbug errors in patch v04. Failed junit tests: TestNameNodeMXBean : passes locally TestLazyPersistReplicaRecovery : passes locally TestDataNodeVolumeFailureReporting : fails in a bunch of builds- HDFS-11488
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 20s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 8s Maven dependency ordering for branch
        +1 mvninstall 13m 59s trunk passed
        +1 compile 1m 34s trunk passed
        +1 checkstyle 0m 44s trunk passed
        +1 mvnsite 1m 43s trunk passed
        +1 mvneclipse 0m 32s trunk passed
        +1 findbugs 3m 36s trunk passed
        +1 javadoc 1m 6s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 1m 28s the patch passed
        +1 compile 1m 48s the patch passed
        +1 javac 1m 48s the patch passed
        +1 checkstyle 0m 48s the patch passed
        +1 mvnsite 2m 0s the patch passed
        +1 mvneclipse 0m 27s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 30s the patch passed
        +1 javadoc 1m 11s the patch passed
        +1 unit 1m 2s hadoop-hdfs-client in the patch passed.
        -1 unit 80m 24s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        119m 27s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.blockmanagement.TestSlowDiskTracker
          hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860413/HDFS-11551.004.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux bd56dbe7fb0e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 0e556a5
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18826/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18826/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18826/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 13m 59s trunk passed +1 compile 1m 34s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 43s trunk passed +1 mvneclipse 0m 32s trunk passed +1 findbugs 3m 36s trunk passed +1 javadoc 1m 6s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 28s the patch passed +1 compile 1m 48s the patch passed +1 javac 1m 48s the patch passed +1 checkstyle 0m 48s the patch passed +1 mvnsite 2m 0s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 30s the patch passed +1 javadoc 1m 11s the patch passed +1 unit 1m 2s hadoop-hdfs-client in the patch passed. -1 unit 80m 24s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 119m 27s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.blockmanagement.TestSlowDiskTracker   hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860413/HDFS-11551.004.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux bd56dbe7fb0e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0e556a5 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18826/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18826/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18826/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Fixed a small error (causing failure in TestSlowDiskTracker) in patch v05.
        The other unit tests pass locally.

        Show
        hanishakoneru Hanisha Koneru added a comment - Fixed a small error (causing failure in TestSlowDiskTracker) in patch v05. The other unit tests pass locally.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 8s Maven dependency ordering for branch
        +1 mvninstall 14m 16s trunk passed
        +1 compile 1m 30s trunk passed
        +1 checkstyle 0m 44s trunk passed
        +1 mvnsite 1m 35s trunk passed
        +1 mvneclipse 0m 29s trunk passed
        +1 findbugs 3m 40s trunk passed
        +1 javadoc 1m 8s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 31s the patch passed
        +1 compile 1m 32s the patch passed
        +1 javac 1m 32s the patch passed
        +1 checkstyle 0m 41s the patch passed
        +1 mvnsite 1m 50s the patch passed
        +1 mvneclipse 0m 27s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 29s the patch passed
        +1 javadoc 1m 11s the patch passed
        +1 unit 1m 9s hadoop-hdfs-client in the patch passed.
        -1 unit 79m 33s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        118m 29s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker
          hadoop.hdfs.server.namenode.TestDecommissioningStatus



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860451/HDFS-11551.005.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 8359977679a6 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d1b7439
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18831/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18831/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18831/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 14m 16s trunk passed +1 compile 1m 30s trunk passed +1 checkstyle 0m 44s trunk passed +1 mvnsite 1m 35s trunk passed +1 mvneclipse 0m 29s trunk passed +1 findbugs 3m 40s trunk passed +1 javadoc 1m 8s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 31s the patch passed +1 compile 1m 32s the patch passed +1 javac 1m 32s the patch passed +1 checkstyle 0m 41s the patch passed +1 mvnsite 1m 50s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 29s the patch passed +1 javadoc 1m 11s the patch passed +1 unit 1m 9s hadoop-hdfs-client in the patch passed. -1 unit 79m 33s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 118m 29s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker   hadoop.hdfs.server.namenode.TestDecommissioningStatus Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860451/HDFS-11551.005.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 8359977679a6 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d1b7439 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18831/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18831/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18831/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Patch v06 uses a different approach for generating slow disk reports. A daemon runs every 30 mins (default, DFS_DATANODE_OUTLIERS_REPORT_INTERVAL_KEY) to generate the top N slow disks from all the reports received from DataNodes. This daemon also cleans up the stale reports.

        Show
        hanishakoneru Hanisha Koneru added a comment - Patch v06 uses a different approach for generating slow disk reports. A daemon runs every 30 mins (default, DFS_DATANODE_OUTLIERS_REPORT_INTERVAL_KEY) to generate the top N slow disks from all the reports received from DataNodes. This daemon also cleans up the stale reports.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 28s Maven dependency ordering for branch
        +1 mvninstall 16m 46s trunk passed
        +1 compile 2m 1s trunk passed
        +1 checkstyle 0m 52s trunk passed
        +1 mvnsite 1m 55s trunk passed
        +1 mvneclipse 0m 31s trunk passed
        +1 findbugs 4m 12s trunk passed
        +1 javadoc 1m 18s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 1m 29s the patch passed
        +1 compile 1m 31s the patch passed
        +1 javac 1m 31s the patch passed
        -0 checkstyle 0m 41s hadoop-hdfs-project: The patch generated 12 new + 44 unchanged - 0 fixed = 56 total (was 44)
        +1 mvnsite 1m 33s the patch passed
        +1 mvneclipse 0m 23s the patch passed
        -1 whitespace 0m 0s The patch 1 line(s) with tabs.
        -1 findbugs 2m 8s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
        +1 javadoc 1m 3s the patch passed
        +1 unit 1m 0s hadoop-hdfs-client in the patch passed.
        -1 unit 78m 30s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        120m 43s



        Reason Tests
        FindBugs module:hadoop-hdfs-project/hadoop-hdfs
          Suspicious comparison of Double references in org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency.areLatenciesEqual(Map) At SlowDiskTracker.java:in org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency.areLatenciesEqual(Map) At SlowDiskTracker.java:[line 213]
        Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.TestSafeModeWithStripedFile



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860774/HDFS-11551.006.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 1f4c0626d9b7 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 9bae672
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/whitespace-tabs.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18859/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18859/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 16m 46s trunk passed +1 compile 2m 1s trunk passed +1 checkstyle 0m 52s trunk passed +1 mvnsite 1m 55s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 4m 12s trunk passed +1 javadoc 1m 18s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 29s the patch passed +1 compile 1m 31s the patch passed +1 javac 1m 31s the patch passed -0 checkstyle 0m 41s hadoop-hdfs-project: The patch generated 12 new + 44 unchanged - 0 fixed = 56 total (was 44) +1 mvnsite 1m 33s the patch passed +1 mvneclipse 0m 23s the patch passed -1 whitespace 0m 0s The patch 1 line(s) with tabs. -1 findbugs 2m 8s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 1m 3s the patch passed +1 unit 1m 0s hadoop-hdfs-client in the patch passed. -1 unit 78m 30s hadoop-hdfs in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 120m 43s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Suspicious comparison of Double references in org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency.areLatenciesEqual(Map) At SlowDiskTracker.java:in org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker$DiskLatency.areLatenciesEqual(Map) At SlowDiskTracker.java: [line 213] Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.TestSafeModeWithStripedFile Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860774/HDFS-11551.006.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 1f4c0626d9b7 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 9bae672 Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/whitespace-tabs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/18859/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18859/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18859/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Fixed checkstyle and findbug errors in patch v07.
        The failed unit tests pass locally.

        Show
        hanishakoneru Hanisha Koneru added a comment - Fixed checkstyle and findbug errors in patch v07. The failed unit tests pass locally.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 21s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 27s Maven dependency ordering for branch
        +1 mvninstall 16m 12s trunk passed
        +1 compile 1m 45s trunk passed
        +1 checkstyle 0m 40s trunk passed
        +1 mvnsite 1m 27s trunk passed
        +1 mvneclipse 0m 26s trunk passed
        +1 findbugs 3m 20s trunk passed
        +1 javadoc 1m 2s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 20s the patch passed
        +1 compile 1m 19s the patch passed
        +1 javac 1m 19s the patch passed
        +1 checkstyle 0m 36s the patch passed
        +1 mvnsite 1m 19s the patch passed
        +1 mvneclipse 0m 21s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 19s the patch passed
        +1 javadoc 0m 57s the patch passed
        +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
        -1 unit 72m 33s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        110m 10s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.balancer.TestBalancer
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860915/HDFS-11551.007.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux ff272ee85927 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6b09336
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18874/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18874/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18874/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 21s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 16m 12s trunk passed +1 compile 1m 45s trunk passed +1 checkstyle 0m 40s trunk passed +1 mvnsite 1m 27s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 20s trunk passed +1 javadoc 1m 2s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 20s the patch passed +1 compile 1m 19s the patch passed +1 javac 1m 19s the patch passed +1 checkstyle 0m 36s the patch passed +1 mvnsite 1m 19s the patch passed +1 mvneclipse 0m 21s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 19s the patch passed +1 javadoc 0m 57s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. -1 unit 72m 33s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 110m 10s Reason Tests Failed junit tests hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12860915/HDFS-11551.007.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ff272ee85927 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6b09336 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18874/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18874/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18874/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Did some optimizations in patch v08.

        Show
        hanishakoneru Hanisha Koneru added a comment - Did some optimizations in patch v08.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 17s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 26s Maven dependency ordering for branch
        +1 mvninstall 14m 6s trunk passed
        +1 compile 1m 31s trunk passed
        +1 checkstyle 0m 42s trunk passed
        +1 mvnsite 1m 32s trunk passed
        +1 mvneclipse 0m 28s trunk passed
        +1 findbugs 3m 39s trunk passed
        +1 javadoc 1m 15s trunk passed
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 1m 39s the patch passed
        +1 compile 1m 50s the patch passed
        +1 javac 1m 50s the patch passed
        -0 checkstyle 0m 43s hadoop-hdfs-project: The patch generated 8 new + 44 unchanged - 0 fixed = 52 total (was 44)
        +1 mvnsite 1m 49s the patch passed
        +1 mvneclipse 0m 26s the patch passed
        -1 whitespace 0m 0s The patch 1 line(s) with tabs.
        -1 findbugs 2m 15s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
        +1 javadoc 1m 5s the patch passed
        +1 unit 1m 2s hadoop-hdfs-client in the patch passed.
        -1 unit 78m 3s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 25s The patch does not generate ASF License warnings.
        117m 3s



        Reason Tests
        FindBugs module:hadoop-hdfs-project/hadoop-hdfs
          Write to static field org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.TIMER from instance method org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.setTimer(Timer) At SlowDiskTracker.java:from instance method org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.setTimer(Timer) At SlowDiskTracker.java:[line 117]
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861137/HDFS-11551.008.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 84fb4ccc956e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / ceab00a
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/whitespace-tabs.txt
        findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18903/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18903/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 26s Maven dependency ordering for branch +1 mvninstall 14m 6s trunk passed +1 compile 1m 31s trunk passed +1 checkstyle 0m 42s trunk passed +1 mvnsite 1m 32s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 39s trunk passed +1 javadoc 1m 15s trunk passed 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 1m 39s the patch passed +1 compile 1m 50s the patch passed +1 javac 1m 50s the patch passed -0 checkstyle 0m 43s hadoop-hdfs-project: The patch generated 8 new + 44 unchanged - 0 fixed = 52 total (was 44) +1 mvnsite 1m 49s the patch passed +1 mvneclipse 0m 26s the patch passed -1 whitespace 0m 0s The patch 1 line(s) with tabs. -1 findbugs 2m 15s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 1m 5s the patch passed +1 unit 1m 2s hadoop-hdfs-client in the patch passed. -1 unit 78m 3s hadoop-hdfs in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 117m 3s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Write to static field org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.TIMER from instance method org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.setTimer(Timer) At SlowDiskTracker.java:from instance method org.apache.hadoop.hdfs.server.blockmanagement.SlowDiskTracker.setTimer(Timer) At SlowDiskTracker.java: [line 117] Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861137/HDFS-11551.008.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 84fb4ccc956e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ceab00a Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/whitespace-tabs.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/18903/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18903/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18903/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Thanks for the updated patch Hanisha Koneru. The v08 patch is looking good. The remaining comments are minor.

        1. isUpgradeInProgress - did you mean to call this isUpdateInProgress?
        2. // Remove is not atomic - I checked the source code for ConcurrentHashMap and it turns out the version of remove you are calling is atomic. I was wrong about the locking behavior earlier.
          http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/concurrent/ConcurrentHashMap.java#ConcurrentHashMap.Segment.remove%28java.lang.Object%2Cint%2Cjava.lang.Object%29

        The test coverage looks good too. Two comments:

        1. The test-only method addSlowDiskForTesting looks unnecessary. Can we just call addSlowDiskReportFromDataNode from the test case?
        2. Also the method getSlowDisksReportForTesting can be moved to the test class - and we can retain a getter for slowDisksReport in the SlowDiskTracker class.
        3. The following code will run a tight loop which will spin the CPU. We can add a sleep() or use GenericTestUtils#waitFor here. Also the call to updateSlowDiskReportAsync can be outside the loop, it just needs to be called once.
              while (slowDisksReport.isEmpty()) {
                slowDiskTracker.updateSlowDiskReportAsync(timer.monotonicNow());
                slowDisksReport = slowDiskTracker.getSlowDisksReportForTesting();
              }
          
        Show
        arpitagarwal Arpit Agarwal added a comment - Thanks for the updated patch Hanisha Koneru . The v08 patch is looking good. The remaining comments are minor. isUpgradeInProgress - did you mean to call this isUpdateInProgress? // Remove is not atomic - I checked the source code for ConcurrentHashMap and it turns out the version of remove you are calling is atomic. I was wrong about the locking behavior earlier. http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/util/concurrent/ConcurrentHashMap.java#ConcurrentHashMap.Segment.remove%28java.lang.Object%2Cint%2Cjava.lang.Object%29 The test coverage looks good too. Two comments: The test-only method addSlowDiskForTesting looks unnecessary. Can we just call addSlowDiskReportFromDataNode from the test case? Also the method getSlowDisksReportForTesting can be moved to the test class - and we can retain a getter for slowDisksReport in the SlowDiskTracker class. The following code will run a tight loop which will spin the CPU. We can add a sleep() or use GenericTestUtils#waitFor here. Also the call to updateSlowDiskReportAsync can be outside the loop, it just needs to be called once. while (slowDisksReport.isEmpty()) { slowDiskTracker.updateSlowDiskReportAsync(timer.monotonicNow()); slowDisksReport = slowDiskTracker.getSlowDisksReportForTesting(); }
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thank you Arpit Agarwal for reviewing the patch and for the comments.
        I have addressed the comments and also checkstyle and findbug errors in patch v09.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal for reviewing the patch and for the comments. I have addressed the comments and also checkstyle and findbug errors in patch v09.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the v09 patch pending Jenkins. Thanks Hanisha Koneru.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the v09 patch pending Jenkins. Thanks Hanisha Koneru .
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 16s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 7s Maven dependency ordering for branch
        +1 mvninstall 14m 0s trunk passed
        +1 compile 1m 35s trunk passed
        +1 checkstyle 0m 43s trunk passed
        +1 mvnsite 1m 36s trunk passed
        +1 mvneclipse 0m 28s trunk passed
        +1 findbugs 3m 37s trunk passed
        +1 javadoc 1m 7s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 27s the patch passed
        +1 compile 1m 31s the patch passed
        +1 javac 1m 31s the patch passed
        -0 checkstyle 0m 42s hadoop-hdfs-project: The patch generated 3 new + 44 unchanged - 0 fixed = 47 total (was 44)
        +1 mvnsite 1m 32s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 49s the patch passed
        +1 javadoc 1m 0s the patch passed
        +1 unit 0m 58s hadoop-hdfs-client in the patch passed.
        -1 unit 70m 43s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 20s The patch does not generate ASF License warnings.
        107m 37s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.blockmanagement.TestSlowDiskTracker



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861338/HDFS-11551.009.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 701fa3e1710e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / b58777a
        Default Java 1.8.0_121
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18914/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18914/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18914/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18914/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 14m 0s trunk passed +1 compile 1m 35s trunk passed +1 checkstyle 0m 43s trunk passed +1 mvnsite 1m 36s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 37s trunk passed +1 javadoc 1m 7s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 27s the patch passed +1 compile 1m 31s the patch passed +1 javac 1m 31s the patch passed -0 checkstyle 0m 42s hadoop-hdfs-project: The patch generated 3 new + 44 unchanged - 0 fixed = 47 total (was 44) +1 mvnsite 1m 32s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 49s the patch passed +1 javadoc 1m 0s the patch passed +1 unit 0m 58s hadoop-hdfs-client in the patch passed. -1 unit 70m 43s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 107m 37s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.blockmanagement.TestSlowDiskTracker Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861338/HDFS-11551.009.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 701fa3e1710e 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / b58777a Default Java 1.8.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18914/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18914/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18914/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18914/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Fixed unit test error in patch v10.

        Show
        hanishakoneru Hanisha Koneru added a comment - Fixed unit test error in patch v10.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 18s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 15m 21s trunk passed
        +1 compile 1m 45s trunk passed
        +1 checkstyle 0m 48s trunk passed
        +1 mvnsite 1m 55s trunk passed
        +1 mvneclipse 0m 31s trunk passed
        +1 findbugs 4m 3s trunk passed
        +1 javadoc 1m 14s trunk passed
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 1m 45s the patch passed
        +1 compile 1m 53s the patch passed
        +1 javac 1m 53s the patch passed
        +1 checkstyle 0m 49s the patch passed
        +1 mvnsite 1m 56s the patch passed
        +1 mvneclipse 0m 29s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 25s the patch passed
        +1 javadoc 1m 12s the patch passed
        +1 unit 1m 8s hadoop-hdfs-client in the patch passed.
        -1 unit 77m 40s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 21s The patch does not generate ASF License warnings.
        119m 40s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11551
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861371/HDFS-11551.010.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b5758b800494 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / bf3fb58
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18921/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18921/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18921/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 15m 21s trunk passed +1 compile 1m 45s trunk passed +1 checkstyle 0m 48s trunk passed +1 mvnsite 1m 55s trunk passed +1 mvneclipse 0m 31s trunk passed +1 findbugs 4m 3s trunk passed +1 javadoc 1m 14s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 45s the patch passed +1 compile 1m 53s the patch passed +1 javac 1m 53s the patch passed +1 checkstyle 0m 49s the patch passed +1 mvnsite 1m 56s the patch passed +1 mvneclipse 0m 29s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 25s the patch passed +1 javadoc 1m 12s the patch passed +1 unit 1m 8s hadoop-hdfs-client in the patch passed. -1 unit 77m 40s hadoop-hdfs in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 119m 40s Reason Tests Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11551 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12861371/HDFS-11551.010.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b5758b800494 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bf3fb58 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18921/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18921/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18921/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        I've committed this to trunk. Thank you for the contribution Hanisha Koneru.

        Made one edit to testDataNodeHeartbeatSlowDiskReport - added try-finally to ensure that the MiniDFSCluster is shutdown.

        Show
        arpitagarwal Arpit Agarwal added a comment - I've committed this to trunk. Thank you for the contribution Hanisha Koneru . Made one edit to testDataNodeHeartbeatSlowDiskReport - added try-finally to ensure that the MiniDFSCluster is shutdown.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11507 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11507/)
        HDFS-11551. Handle SlowDiskReport from DataNode at the NameNode. (arp: rev 28cdc5a8dc37ade1f45bda3aede589ee8593945e)

        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeDiskMetrics.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SlowDiskReports.java
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSlowDiskTracker.java
        • (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowDiskTracker.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11507 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11507/ ) HDFS-11551 . Handle SlowDiskReport from DataNode at the NameNode. (arp: rev 28cdc5a8dc37ade1f45bda3aede589ee8593945e) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/metrics/DataNodeDiskMetrics.java (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/server/protocol/SlowDiskReports.java (add) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestSlowDiskTracker.java (add) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SlowDiskTracker.java
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        Also I reran the two failed UTs locally and they passed.

        Show
        arpitagarwal Arpit Agarwal added a comment - Also I reran the two failed UTs locally and they passed.
        Hide
        hanishakoneru Hanisha Koneru added a comment -

        Thank you Arpit Agarwal for committing the patch.

        Show
        hanishakoneru Hanisha Koneru added a comment - Thank you Arpit Agarwal for committing the patch.
        Hide
        arpitagarwal Arpit Agarwal added a comment -

        +1 for the v2 branch-2 patch. I've committed it. Thanks Hanisha Koneru.

        Show
        arpitagarwal Arpit Agarwal added a comment - +1 for the v2 branch-2 patch. I've committed it. Thanks Hanisha Koneru .

          People

          • Assignee:
            hanishakoneru Hanisha Koneru
            Reporter:
            hanishakoneru Hanisha Koneru
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development