Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.9.0, 3.0.0-alpha2
    • Component/s: datanode, namenode
    • Labels:
      None
    1. HDFS-11259.01.patch
      19 kB
      Manoj Govindassamy
    2. HDFS-11259.02.patch
      23 kB
      Manoj Govindassamy
    3. HDFS-11259.03.patch
      28 kB
      Manoj Govindassamy
    4. HDFS-11259.04.patch
      28 kB
      Manoj Govindassamy
    5. HDFS-11259-branch-2.01.patch
      23 kB
      Manoj Govindassamy
    6. HDFS-11259-branch-2.02.patch
      24 kB
      Manoj Govindassamy
    7. HDFS-11259-branch-2.03.patch
      24 kB
      Manoj Govindassamy

      Activity

      Hide
      manojg Manoj Govindassamy added a comment -

      Attached v01 patch to address the following:
      1. Made NamenodeFsck to gather details on maintenance replicas for the following checks and modules – blockIdCk(), getReplicaInfo(), collectBlockSummary() and Result

      2. Updated TestFsck to verify maintenance replicas for the following commands and tests

      • hdfs fsck / (testFsckWithMaintenanceReplicas)
      • hdfs fsck / -files -blocks -replicaDetails (testFsckReplicaDetails())
      • hdfs fsck -blockId blockid (testBlockIdCKMaintenance())
        PS: TestFsck changes might show checkstyle issues for not using private variables. I chose to follow the usage pattern of all variables in Result class. May be I can work on cleaning up the checkstyle issues for the whole TestFsck later.
        Lei (Eddy) Xu, Ming Ma, can you please review the patch and let me know your comments ?
      Show
      manojg Manoj Govindassamy added a comment - Attached v01 patch to address the following: 1. Made NamenodeFsck to gather details on maintenance replicas for the following checks and modules – blockIdCk(), getReplicaInfo(), collectBlockSummary() and Result 2. Updated TestFsck to verify maintenance replicas for the following commands and tests hdfs fsck / (testFsckWithMaintenanceReplicas) hdfs fsck / -files -blocks -replicaDetails (testFsckReplicaDetails()) hdfs fsck -blockId blockid (testBlockIdCKMaintenance()) PS: TestFsck changes might show checkstyle issues for not using private variables. I chose to follow the usage pattern of all variables in Result class. May be I can work on cleaning up the checkstyle issues for the whole TestFsck later. Lei (Eddy) Xu , Ming Ma , can you please review the patch and let me know your comments ?
      Hide
      eddyxu Lei (Eddy) Xu added a comment -

      Hi, Manoj Govindassamy

      Thanks for working on it. LGTM. +1.

      I will wait a day for further comments.

      Show
      eddyxu Lei (Eddy) Xu added a comment - Hi, Manoj Govindassamy Thanks for working on it. LGTM. +1. I will wait a day for further comments.
      Hide
      dilaver Dilaver added a comment -

      Thanks for the change Manoj. LGTM.
      Some nits around test polling and timeouts:

      +    do {
      +      Thread.sleep(2000);
      +      for (DatanodeInfo info : dfs.getDataNodeStats()) {
      +        if (dnName.equals(info.getXferAddr())) {
      +          datanodeInfo = info;
      +        }
      +      }
      +    } while (datanodeInfo != null && !datanodeInfo.isInMaintenance());
      
      • please check if something like GenericTestUtils#waitFor can be used where the state is being polled, or if refactoring them into something like "waitForInMaintenance" makes sense
      • consider sleeping less than 2sec while polling
      • also consider adding timeouts to test cases
      Show
      dilaver Dilaver added a comment - Thanks for the change Manoj. LGTM. Some nits around test polling and timeouts: + do { + Thread .sleep(2000); + for (DatanodeInfo info : dfs.getDataNodeStats()) { + if (dnName.equals(info.getXferAddr())) { + datanodeInfo = info; + } + } + } while (datanodeInfo != null && !datanodeInfo.isInMaintenance()); please check if something like GenericTestUtils#waitFor can be used where the state is being polled, or if refactoring them into something like "waitForInMaintenance" makes sense consider sleeping less than 2sec while polling also consider adding timeouts to test cases
      Hide
      manojg Manoj Govindassamy added a comment -

      Thanks Lei (Eddy) Xu and Dilaver for the review. Attached v02 patch incorporating test review comments.

      Show
      manojg Manoj Govindassamy added a comment - Thanks Lei (Eddy) Xu and Dilaver for the review. Attached v02 patch incorporating test review comments.
      Hide
      hadoopqa Hadoop QA added a comment -
      +1 overall



      Vote Subsystem Runtime Comment
      0 reexec 0m 20s Docker mode activated.
      +1 @author 0m 1s The patch does not contain any @author tags.
      +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
      +1 mvninstall 16m 10s trunk passed
      +1 compile 0m 59s trunk passed
      +1 checkstyle 0m 27s trunk passed
      +1 mvnsite 0m 53s trunk passed
      +1 mvneclipse 0m 13s trunk passed
      +1 findbugs 1m 47s trunk passed
      +1 javadoc 0m 40s trunk passed
      +1 mvninstall 0m 46s the patch passed
      +1 compile 0m 44s the patch passed
      +1 javac 0m 44s the patch passed
      -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 130 unchanged - 1 fixed = 133 total (was 131)
      +1 mvnsite 0m 50s the patch passed
      +1 mvneclipse 0m 11s the patch passed
      +1 whitespace 0m 0s The patch has no whitespace issues.
      +1 findbugs 1m 54s the patch passed
      +1 javadoc 0m 37s the patch passed
      +1 unit 64m 3s hadoop-hdfs in the patch passed.
      +1 asflicense 0m 19s The patch does not generate ASF License warnings.
      92m 34s



      Subsystem Report/Notes
      Docker Image:yetus/hadoop:a9ad5d6
      JIRA Issue HDFS-11259
      JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846145/HDFS-11259.02.patch
      Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
      uname Linux 0f00ec9b4204 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
      Build tool maven
      Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
      git revision trunk / 0203164
      Default Java 1.8.0_111
      findbugs v3.0.0
      checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18101/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
      Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18101/testReport/
      modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
      Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18101/console
      Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 @author 0m 1s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 16m 10s trunk passed +1 compile 0m 59s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 40s trunk passed +1 mvninstall 0m 46s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 130 unchanged - 1 fixed = 133 total (was 131) +1 mvnsite 0m 50s the patch passed +1 mvneclipse 0m 11s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 54s the patch passed +1 javadoc 0m 37s the patch passed +1 unit 64m 3s hadoop-hdfs in the patch passed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 92m 34s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11259 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12846145/HDFS-11259.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0f00ec9b4204 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0203164 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18101/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18101/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18101/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
      Hide
      eddyxu Lei (Eddy) Xu added a comment -

      +1. The latest patch LGTM.

      Committed to 3.0-alpha2

      Thanks Manoj Govindassamy for reporting and working on the patch.

      Show
      eddyxu Lei (Eddy) Xu added a comment - +1. The latest patch LGTM. Committed to 3.0-alpha2 Thanks Manoj Govindassamy for reporting and working on the patch.
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11100 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11100/)
      HDFS-11259. Update fsck to display maintenance state info. (Manoj (lei: rev c18590fce283378edb09acd4e764706a9a4a8b5f)

      • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
      • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
      • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11100 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11100/ ) HDFS-11259 . Update fsck to display maintenance state info. (Manoj (lei: rev c18590fce283378edb09acd4e764706a9a4a8b5f) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
      Hide
      manojg Manoj Govindassamy added a comment -

      Lei (Eddy) Xu Attached branch-2 patch. Can you please review the same ? Previously this jira has been marked as incompatible and not sure how we should treat this for the branch-2 backport.

      Show
      manojg Manoj Govindassamy added a comment - Lei (Eddy) Xu Attached branch-2 patch. Can you please review the same ? Previously this jira has been marked as incompatible and not sure how we should treat this for the branch-2 backport.
      Hide
      andrew.wang Andrew Wang added a comment -

      Manoj Govindassamy could you add a release note for this incompatible JIRA? Thanks!

      Show
      andrew.wang Andrew Wang added a comment - Manoj Govindassamy could you add a release note for this incompatible JIRA? Thanks!
      Hide
      andrew.wang Andrew Wang added a comment -

      I took a quick look at this patch, it looks like it unconditionally prints out maintenance info in fsck, which is incompatible.

      We can make it compatible by hiding it behind a command line flag or a config option. It'd also be nice to keep the behavior of branch-2 and trunk the same, so I'd consider reverting this from trunk for now.

      Show
      andrew.wang Andrew Wang added a comment - I took a quick look at this patch, it looks like it unconditionally prints out maintenance info in fsck, which is incompatible. We can make it compatible by hiding it behind a command line flag or a config option. It'd also be nice to keep the behavior of branch-2 and trunk the same, so I'd consider reverting this from trunk for now.
      Hide
      eddyxu Lei (Eddy) Xu added a comment -

      Make sense Andrew Wang. Manoj Govindassamy would you mind to add a flag to display maintenance info to both branches?

      I'm going to revert trunk for now.

      Show
      eddyxu Lei (Eddy) Xu added a comment - Make sense Andrew Wang . Manoj Govindassamy would you mind to add a flag to display maintenance info to both branches? I'm going to revert trunk for now.
      Hide
      hudson Hudson added a comment -

      SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11118 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11118/)
      Revert "HDFS-11259. Update fsck to display maintenance state info. (lei: rev 2f8e9b7e4b1721ed5c7db8882eff70f83164e320)

      • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
      • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
      • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
      Show
      hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11118 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11118/ ) Revert " HDFS-11259 . Update fsck to display maintenance state info. (lei: rev 2f8e9b7e4b1721ed5c7db8882eff70f83164e320) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSck.java
      Hide
      manojg Manoj Govindassamy added a comment -

      Thanks for the review Andrew Wang and Lei (Eddy) Xu.

      Adding an option for including node details on maintenance state does sound like a good idea and helps in making the patch backward compatible. Attached v03 patch, to address the following

      • Added an option -includeMaintenanceState to fsck command to print maintenance state related node details
      • Updated HDFSCommands.md to include help for the new option in fsck command
      • Updated TestFsck to verify fsck results with and without the new command option

      PS: Patch might have checkstyle issues w.r.t function length, and on non private variables. All these are already existing ones and can be fixed outside of this bug.

      Lei (Eddy) Xu, can you please review the patch ? Once I hear your comments on this, can extend the patch to branch-2 as well. Please let me know.

      Show
      manojg Manoj Govindassamy added a comment - Thanks for the review Andrew Wang and Lei (Eddy) Xu . Adding an option for including node details on maintenance state does sound like a good idea and helps in making the patch backward compatible. Attached v03 patch, to address the following Added an option -includeMaintenanceState to fsck command to print maintenance state related node details Updated HDFSCommands.md to include help for the new option in fsck command Updated TestFsck to verify fsck results with and without the new command option PS: Patch might have checkstyle issues w.r.t function length, and on non private variables. All these are already existing ones and can be fixed outside of this bug. Lei (Eddy) Xu , can you please review the patch ? Once I hear your comments on this, can extend the patch to branch-2 as well. Please let me know.
      Hide
      manojg Manoj Govindassamy added a comment -

      Reopening the issue as the old backward-incompatible fix has been reverted.

      Show
      manojg Manoj Govindassamy added a comment - Reopening the issue as the old backward-incompatible fix has been reverted.
      Hide
      manojg Manoj Govindassamy added a comment -

      Removing the 'incompatible changes' flag as a new option has been introduced for the fsck command and the output format will include newer maintenance state related details only when requested.

      Show
      manojg Manoj Govindassamy added a comment - Removing the 'incompatible changes' flag as a new option has been introduced for the fsck command and the output format will include newer maintenance state related details only when requested.
      Hide
      hadoopqa Hadoop QA added a comment -
      -1 overall



      Vote Subsystem Runtime Comment
      0 reexec 13m 16s Docker mode activated.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
      +1 mvninstall 7m 16s branch-2 passed
      +1 compile 0m 50s branch-2 passed with JDK v1.8.0_121
      +1 compile 0m 44s branch-2 passed with JDK v1.7.0_121
      +1 checkstyle 0m 30s branch-2 passed
      +1 mvnsite 0m 55s branch-2 passed
      +1 mvneclipse 0m 13s branch-2 passed
      +1 findbugs 1m 52s branch-2 passed
      +1 javadoc 0m 55s branch-2 passed with JDK v1.8.0_121
      +1 javadoc 1m 34s branch-2 passed with JDK v1.7.0_121
      +1 mvninstall 0m 43s the patch passed
      +1 compile 0m 40s the patch passed with JDK v1.8.0_121
      +1 javac 0m 40s the patch passed
      +1 compile 0m 41s the patch passed with JDK v1.7.0_121
      +1 javac 0m 41s the patch passed
      -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 134 unchanged - 1 fixed = 138 total (was 135)
      +1 mvnsite 0m 51s the patch passed
      +1 mvneclipse 0m 12s the patch passed
      +1 whitespace 0m 0s The patch has no whitespace issues.
      +1 findbugs 2m 8s the patch passed
      +1 javadoc 0m 55s the patch passed with JDK v1.8.0_121
      +1 javadoc 1m 37s the patch passed with JDK v1.7.0_121
      -1 unit 55m 29s hadoop-hdfs in the patch failed with JDK v1.7.0_121.
      +1 asflicense 0m 27s The patch does not generate ASF License warnings.
      152m 38s



      Reason Tests
      JDK v1.8.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
      JDK v1.7.0_121 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
      JDK v1.7.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



      Subsystem Report/Notes
      Docker Image:yetus/hadoop:b59b8b7
      JIRA Issue HDFS-11259
      JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848139/HDFS-11259-branch-2.02.patch
      Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
      uname Linux b479c3b40ea1 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
      Build tool maven
      Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
      git revision branch-2 / e2b3eff
      Default Java 1.7.0_121
      Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121
      findbugs v3.0.0
      checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18195/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
      unit https://builds.apache.org/job/PreCommit-HDFS-Build/18195/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt
      JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18195/testReport/
      modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
      Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18195/console
      Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 16s branch-2 passed +1 compile 0m 50s branch-2 passed with JDK v1.8.0_121 +1 compile 0m 44s branch-2 passed with JDK v1.7.0_121 +1 checkstyle 0m 30s branch-2 passed +1 mvnsite 0m 55s branch-2 passed +1 mvneclipse 0m 13s branch-2 passed +1 findbugs 1m 52s branch-2 passed +1 javadoc 0m 55s branch-2 passed with JDK v1.8.0_121 +1 javadoc 1m 34s branch-2 passed with JDK v1.7.0_121 +1 mvninstall 0m 43s the patch passed +1 compile 0m 40s the patch passed with JDK v1.8.0_121 +1 javac 0m 40s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_121 +1 javac 0m 41s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 134 unchanged - 1 fixed = 138 total (was 135) +1 mvnsite 0m 51s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 8s the patch passed +1 javadoc 0m 55s the patch passed with JDK v1.8.0_121 +1 javadoc 1m 37s the patch passed with JDK v1.7.0_121 -1 unit 55m 29s hadoop-hdfs in the patch failed with JDK v1.7.0_121. +1 asflicense 0m 27s The patch does not generate ASF License warnings. 152m 38s Reason Tests JDK v1.8.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting JDK v1.7.0_121 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs JDK v1.7.0_121 Timed out junit tests org.apache.hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Image:yetus/hadoop:b59b8b7 JIRA Issue HDFS-11259 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848139/HDFS-11259-branch-2.02.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b479c3b40ea1 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / e2b3eff Default Java 1.7.0_121 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_121 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_121 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18195/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18195/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_121.txt JDK v1.7.0_121 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18195/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18195/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
      Hide
      manojg Manoj Govindassamy added a comment -

      I don't see the failed tests related to the patch and they are passing through locally for me.

      Show
      manojg Manoj Govindassamy added a comment - I don't see the failed tests related to the patch and they are passing through locally for me.
      Hide
      eddyxu Lei (Eddy) Xu added a comment -

      Hi, Manoj Govindassamy.

      LGTM overall. One small nit:

      You might want to consider to change "-includeMaintenanceState" to "-maintenance", to be consistent with the rest of flags, which are nouns. You might also want to provide a tests without "-maintenance" flag in fsck, and verify that the outputs do not contain maintenance state.

      assertTrue(!fsckOut.contains("(ENTERING MAINTENANCE)"));
      assertTrue(!fsckOut.contains("(IN MAINTENANCE)"));
      

      assertFalse?

      Thanks!

      Show
      eddyxu Lei (Eddy) Xu added a comment - Hi, Manoj Govindassamy . LGTM overall. One small nit: You might want to consider to change "-includeMaintenanceState" to "-maintenance", to be consistent with the rest of flags, which are nouns. You might also want to provide a tests without "-maintenance" flag in fsck, and verify that the outputs do not contain maintenance state. assertTrue(!fsckOut.contains( "(ENTERING MAINTENANCE)" )); assertTrue(!fsckOut.contains( "(IN MAINTENANCE)" )); assertFalse ? Thanks!
      Hide
      manojg Manoj Govindassamy added a comment -

      Thanks for the review Lei (Eddy) Xu. Attached new set of patches for trunk and branch-2 which addresses the following

      • Changed command line option from "includeMaintenanceState" to "maintenance"
      • Updated TestFsck with new command line options, used assertFalse. Test already has positive/negative cases for "-maintenance" command line option.
        Kindly take a look.
      Show
      manojg Manoj Govindassamy added a comment - Thanks for the review Lei (Eddy) Xu . Attached new set of patches for trunk and branch-2 which addresses the following Changed command line option from "includeMaintenanceState" to "maintenance" Updated TestFsck with new command line options, used assertFalse. Test already has positive/negative cases for "-maintenance" command line option. Kindly take a look.
      Hide
      eddyxu Lei (Eddy) Xu added a comment -

      +1.

      Committed to trunk and branch-2.

      Thanks a lot!

      Show
      eddyxu Lei (Eddy) Xu added a comment - +1. Committed to trunk and branch-2. Thanks a lot!
      Hide
      hadoopqa Hadoop QA added a comment -
      -1 overall



      Vote Subsystem Runtime Comment
      0 reexec 0m 14s Docker mode activated.
      +1 @author 0m 0s The patch does not contain any @author tags.
      +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
      +1 mvninstall 14m 49s trunk passed
      +1 compile 0m 56s trunk passed
      +1 checkstyle 0m 31s trunk passed
      +1 mvnsite 1m 3s trunk passed
      +1 mvneclipse 0m 14s trunk passed
      +1 findbugs 2m 2s trunk passed
      +1 javadoc 0m 45s trunk passed
      +1 mvninstall 0m 56s the patch passed
      +1 compile 0m 52s the patch passed
      +1 javac 0m 52s the patch passed
      -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 130 unchanged - 1 fixed = 134 total (was 131)
      +1 mvnsite 0m 58s the patch passed
      +1 mvneclipse 0m 12s the patch passed
      +1 whitespace 0m 0s The patch has no whitespace issues.
      +1 findbugs 2m 10s the patch passed
      +1 javadoc 0m 41s the patch passed
      -1 unit 76m 15s hadoop-hdfs in the patch failed.
      +1 asflicense 0m 33s The patch does not generate ASF License warnings.
      105m 4s



      Reason Tests
      Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



      Subsystem Report/Notes
      Docker Image:yetus/hadoop:a9ad5d6
      JIRA Issue HDFS-11259
      JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848243/HDFS-11259.04.patch
      Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
      uname Linux 4a603f2b89ce 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
      Build tool maven
      Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
      git revision trunk / 724bb86
      Default Java 1.8.0_111
      findbugs v3.0.0
      checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18207/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
      unit https://builds.apache.org/job/PreCommit-HDFS-Build/18207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
      Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18207/testReport/
      modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
      Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18207/console
      Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

      This message was automatically generated.

      Show
      hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 14m 49s trunk passed +1 compile 0m 56s trunk passed +1 checkstyle 0m 31s trunk passed +1 mvnsite 1m 3s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 2m 2s trunk passed +1 javadoc 0m 45s trunk passed +1 mvninstall 0m 56s the patch passed +1 compile 0m 52s the patch passed +1 javac 0m 52s the patch passed -0 checkstyle 0m 28s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 130 unchanged - 1 fixed = 134 total (was 131) +1 mvnsite 0m 58s the patch passed +1 mvneclipse 0m 12s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 10s the patch passed +1 javadoc 0m 41s the patch passed -1 unit 76m 15s hadoop-hdfs in the patch failed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 105m 4s Reason Tests Timed out junit tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11259 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848243/HDFS-11259.04.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4a603f2b89ce 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 724bb86 Default Java 1.8.0_111 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/18207/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/18207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18207/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18207/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

        People

        • Assignee:
          manojg Manoj Govindassamy
          Reporter:
          manojg Manoj Govindassamy
        • Votes:
          0 Vote for this issue
          Watchers:
          7 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development