Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9231

fsck doesn't list correct file path when Bad Replicas/Blocks are in a snapshot

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Currently for snapshot files, fsck -list-corruptfileblocks shows corrupt blocks with the original file dir instead of the snapshot dir, and fsck -list-corruptfileblocks -includeSnapshots behave the same.
      This can be confusing because even when the original file is deleted, fsck will still show that deleted file as corrupted, although what's actually corrupted is the snapshot.

      As a side note, fsck -files -includeSnapshots shows the snapshot dirs.

      1. HDFS-9231.001.patch
        12 kB
        Xiao Chen
      2. HDFS-9231.002.patch
        12 kB
        Xiao Chen
      3. HDFS-9231.003.patch
        12 kB
        Xiao Chen
      4. HDFS-9231.004.patch
        13 kB
        Xiao Chen
      5. HDFS-9231.005.patch
        14 kB
        Xiao Chen
      6. HDFS-9231.006.patch
        15 kB
        Xiao Chen
      7. HDFS-9231.007.patch
        16 kB
        Xiao Chen
      8. HDFS-9231.008.patch
        16 kB
        Xiao Chen
      9. HDFS-9231.009.patch
        16 kB
        Xiao Chen

        Issue Links

          Activity

          Hide
          xiaochen Xiao Chen added a comment -

          Attached patch 001. Description below:

          • fsck from command line with -includeSnapshots will also show full dir of snapshots
          • fsck from command line without -includeSnapshots behavior unchanged
          • NameNode WebUI always show full dir of snapshots
          • Some refactoring to reuse getSnapshottableDirs and ListCorruptFileBlocksWithSnapshot
          • Getting all possible snapshots is not so efficient, but considering fsck should not be performed frequently and nothing is added into listCorruptFileBlocks where fslock is used, the impact should be minimal.
          Show
          xiaochen Xiao Chen added a comment - Attached patch 001. Description below: fsck from command line with -includeSnapshots will also show full dir of snapshots fsck from command line without -includeSnapshots behavior unchanged NameNode WebUI always show full dir of snapshots Some refactoring to reuse getSnapshottableDirs and ListCorruptFileBlocksWithSnapshot Getting all possible snapshots is not so efficient, but considering fsck should not be performed frequently and nothing is added into listCorruptFileBlocks where fslock is used, the impact should be minimal.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 18m 36s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 4s There were no new javac warning messages.
          +1 javadoc 10m 34s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 56s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 40s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 27s Pre-build of native portion
          -1 hdfs tests 50m 18s Tests failed in hadoop-hdfs.
              97m 10s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.namenode.TestFSNamesystemMBean



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766475/HDFS-9231.001.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 2a98724
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12971/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12971/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12971/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12971/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 36s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 4s There were no new javac warning messages. +1 javadoc 10m 34s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 56s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 40s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 32s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 27s Pre-build of native portion -1 hdfs tests 50m 18s Tests failed in hadoop-hdfs.     97m 10s   Reason Tests Failed unit tests hadoop.hdfs.server.namenode.TestFSNamesystemMBean Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766475/HDFS-9231.001.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 2a98724 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12971/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12971/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12971/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12971/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Patch 002 addresses the test failure. The findbugs warning seems odd.

          Show
          xiaochen Xiao Chen added a comment - Patch 002 addresses the test failure. The findbugs warning seems odd.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 18m 11s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 54s There were no new javac warning messages.
          +1 javadoc 10m 21s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 22s The applied patch generated 4 new checkstyle issues (total was 370, now 372).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse.
          +1 findbugs 2m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 11s Pre-build of native portion
          -1 hdfs tests 49m 52s Tests failed in hadoop-hdfs.
              96m 2s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000
            hadoop.hdfs.TestEncryptionZonesWithKMS
            hadoop.hdfs.server.blockmanagement.TestBlockManager
            hadoop.fs.TestGlobPaths
            hadoop.hdfs.TestReplaceDatanodeOnFailure



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766570/HDFS-9231.002.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 0d77e85
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12981/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12981/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12981/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12981/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12981/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 11s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 54s There were no new javac warning messages. +1 javadoc 10m 21s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 22s The applied patch generated 4 new checkstyle issues (total was 370, now 372). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 38s The patch built with eclipse:eclipse. +1 findbugs 2m 35s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 11s Pre-build of native portion -1 hdfs tests 49m 52s Tests failed in hadoop-hdfs.     96m 2s   Reason Tests Failed unit tests hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000   hadoop.hdfs.TestEncryptionZonesWithKMS   hadoop.hdfs.server.blockmanagement.TestBlockManager   hadoop.fs.TestGlobPaths   hadoop.hdfs.TestReplaceDatanodeOnFailure Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766570/HDFS-9231.002.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 0d77e85 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12981/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12981/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12981/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12981/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12981/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Patch 003 addresses 1 checkstyle warning that's related.
          Other warning/errors are unrelated.

          Show
          xiaochen Xiao Chen added a comment - Patch 003 addresses 1 checkstyle warning that's related. Other warning/errors are unrelated.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 18m 11s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 6s There were no new javac warning messages.
          +1 javadoc 10m 27s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 26s The applied patch generated 3 new checkstyle issues (total was 370, now 371).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse.
          +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 25s Pre-build of native portion
          -1 hdfs tests 50m 13s Tests failed in hadoop-hdfs.
              96m 48s  



          Reason Tests
          Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.fs.TestGlobPaths



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12766629/HDFS-9231.003.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 3d50855
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12990/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12990/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12990/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12990/testReport/
          Java 1.7.0_55
          uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12990/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 18m 11s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 6s There were no new javac warning messages. +1 javadoc 10m 27s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 26s The applied patch generated 3 new checkstyle issues (total was 370, now 371). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 33s The patch built with eclipse:eclipse. +1 findbugs 2m 30s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 25s Pre-build of native portion -1 hdfs tests 50m 13s Tests failed in hadoop-hdfs.     96m 48s   Reason Tests Failed unit tests hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.fs.TestGlobPaths Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12766629/HDFS-9231.003.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 3d50855 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/12990/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12990/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12990/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12990/testReport/ Java 1.7.0_55 uname Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12990/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          The pre-patch Findbugs warning is unrelated. (see HDFS-9242)
          The checkstyle issues are not introduced by this patch.
          The test failures are unrelated, and passed locally.

          Show
          xiaochen Xiao Chen added a comment - The pre-patch Findbugs warning is unrelated. (see HDFS-9242 ) The checkstyle issues are not introduced by this patch. The test failures are unrelated, and passed locally.
          Hide
          xiaochen Xiao Chen added a comment -

          patch 004 is attached. I think I should re-summarize it below:

          • fsck from command line with -includeSnapshots will also show full dir of snapshots
          • fsck from command line without -includeSnapshots behavior unchanged
          • NameNode WebUI's way of showing corrupted files/blocks unchanged.
          • Added a sentence in NN WebUI to hint the admin to run fsck with -includeSnapshots if there're snapshots present in the system.
          • Some refactoring to reuse existing code in new methods getSnapshottableDirs and ListCorruptFileBlocksWithSnapshot
          • The reasoning of keep minimal change to NN WebUI and fsck without -includeSnapshots is that getting all possible snapshots may be slow, since it's user configured.
          Show
          xiaochen Xiao Chen added a comment - patch 004 is attached. I think I should re-summarize it below: fsck from command line with -includeSnapshots will also show full dir of snapshots fsck from command line without -includeSnapshots behavior unchanged NameNode WebUI's way of showing corrupted files/blocks unchanged. Added a sentence in NN WebUI to hint the admin to run fsck with -includeSnapshots if there're snapshots present in the system. Some refactoring to reuse existing code in new methods getSnapshottableDirs and ListCorruptFileBlocksWithSnapshot The reasoning of keep minimal change to NN WebUI and fsck without -includeSnapshots is that getting all possible snapshots may be slow, since it's user configured.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 16m 10s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 58s There were no new javac warning messages.
          +1 javadoc 10m 30s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 35s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 36s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          -1 findbugs 2m 34s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 native 3m 13s Pre-build of native portion
          -1 hdfs tests 51m 8s Tests failed in hadoop-hdfs.
              94m 47s  



          Reason Tests
          FindBugs module:hadoop-hdfs
          Failed unit tests hadoop.hdfs.tools.TestDFSHAAdminMiniCluster
            hadoop.hdfs.tools.TestStoragePolicyCommands
            hadoop.hdfs.tools.TestDFSZKFailoverController
            hadoop.hdfs.TestRecoverStripedFile
            hadoop.hdfs.server.namenode.ha.TestDNFencing
          Timed out tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12767823/HDFS-9231.004.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / e27c2ae
          Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13110/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13110/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13110/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13110/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 16m 10s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 58s There were no new javac warning messages. +1 javadoc 10m 30s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 35s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 36s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. -1 findbugs 2m 34s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 13s Pre-build of native portion -1 hdfs tests 51m 8s Tests failed in hadoop-hdfs.     94m 47s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.tools.TestDFSHAAdminMiniCluster   hadoop.hdfs.tools.TestStoragePolicyCommands   hadoop.hdfs.tools.TestDFSZKFailoverController   hadoop.hdfs.TestRecoverStripedFile   hadoop.hdfs.server.namenode.ha.TestDNFencing Timed out tests org.apache.hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12767823/HDFS-9231.004.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / e27c2ae Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13110/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13110/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13110/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13110/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          The test failures look unrelated and passed locally.
          Findbugs seems to be having problem.

          Exception in thread "main" java.io.FileNotFoundException: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/trunkFindbugsWarningshadoop-hdfs.xml (No such file or directory)
          	at java.io.FileInputStream.open(Native Method)
          	at java.io.FileInputStream.<init>(FileInputStream.java:146)
          	at edu.umd.cs.findbugs.SortedBugCollection.progessMonitoredInputStream(SortedBugCollection.java:1231)
          	at edu.umd.cs.findbugs.SortedBugCollection.readXML(SortedBugCollection.java:308)
          	at edu.umd.cs.findbugs.SortedBugCollection.readXML(SortedBugCollection.java:295)
          	at edu.umd.cs.findbugs.workflow.Filter.main(Filter.java:712)
          Pre-patch trunk findbugs is broken?
          
          Show
          xiaochen Xiao Chen added a comment - The test failures look unrelated and passed locally. Findbugs seems to be having problem. Exception in thread "main" java.io.FileNotFoundException: /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/trunkFindbugsWarningshadoop-hdfs.xml (No such file or directory) at java.io.FileInputStream.open(Native Method) at java.io.FileInputStream.<init>(FileInputStream.java:146) at edu.umd.cs.findbugs.SortedBugCollection.progessMonitoredInputStream(SortedBugCollection.java:1231) at edu.umd.cs.findbugs.SortedBugCollection.readXML(SortedBugCollection.java:308) at edu.umd.cs.findbugs.SortedBugCollection.readXML(SortedBugCollection.java:295) at edu.umd.cs.findbugs.workflow.Filter.main(Filter.java:712) Pre-patch trunk findbugs is broken?
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Xiao Chen,

          Thanks for reporting the issue and the patch. Good to have discussed in person. The patch looks good in general. I have some comments here:

          1. The description is not quite accurate per our discussion, suggest to modify. Especially the patch actually does change (and fix) the behavior when without -includeSnapshots.
          2. A possible optimization in FSDirSnapshotOp#getSnapshotFiles. It seems that the sf variable could be calculated in caller for once before the loop in the caller, and pass to this method.
          3. final INodesInPath iip = fsd.getINodesInPath4Write(snap, false); maybe substituted with call to getINodesInPath
          4. The check if (!corruptFileBlocks.isEmpty()) in listCorruptFileBlocksWithSnapshot is not needed
          5. Add comment in listCorruptFileBlocks() before the call namenode.getNamesystem().listCorruptFileBlocksWithSnapshot, to indicate that snapshottableDirs is only relevant when -includeSnapshots is specified.
          6. In listCorruptFileBlocksWithSnapshot, we can add

          if (snapshottableDirs == null) {
            continue;
          }
          

          to avoid the call to getSnapshotFiles.

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Xiao Chen , Thanks for reporting the issue and the patch. Good to have discussed in person. The patch looks good in general. I have some comments here: 1. The description is not quite accurate per our discussion, suggest to modify. Especially the patch actually does change (and fix) the behavior when without -includeSnapshots. 2. A possible optimization in FSDirSnapshotOp#getSnapshotFiles. It seems that the sf variable could be calculated in caller for once before the loop in the caller, and pass to this method. 3. final INodesInPath iip = fsd.getINodesInPath4Write(snap, false); maybe substituted with call to getINodesInPath 4. The check if (!corruptFileBlocks.isEmpty()) in listCorruptFileBlocksWithSnapshot is not needed 5. Add comment in listCorruptFileBlocks() before the call namenode.getNamesystem().listCorruptFileBlocksWithSnapshot , to indicate that snapshottableDirs is only relevant when -includeSnapshots is specified. 6. In listCorruptFileBlocksWithSnapshot , we can add if (snapshottableDirs == null ) { continue ; } to avoid the call to getSnapshotFiles . Thanks.
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks a lot for the review Yongjun Zhang!

          1. The description is not quite accurate per our discussion, suggest to modify. Especially the patch actually does change (and fix) the behavior when without -includeSnapshots.

          It was great to talk to you. I have updated the description. Modified patch summary in the end of this comment.

          2. A possible optimization in FSDirSnapshotOp#getSnapshotFiles. It seems that the sf variable could be calculated in caller for once before the loop in the caller, and pass to this method.

          My apologies for the confusion, I added some comments in this method. But getting sf for each snapshottable dir is needed, since /d1 and /d2 have different snapshotlist.

          3. final INodesInPath iip = fsd.getINodesInPath4Write(snap, false); maybe substituted with call to getINodesInPath

          Good catch! I updated the code to call getINode which invokes getINodesInPath.

          4. The check if (!corruptFileBlocks.isEmpty()) in listCorruptFileBlocksWithSnapshot is not needed

          Good call. Fixed.

          5. Add comment in listCorruptFileBlocks() before the call namenode.getNamesystem().listCorruptFileBlocksWithSnapshot, to indicate that snapshottableDirs is only relevant when -includeSnapshots is specified.

          Added a link to FSNamesystem#listCorruptFileBlocksWithSnapshot which explains that parameter in javadoc.

          6. In listCorruptFileBlocksWithSnapshot, we can add

          if (snapshottableDirs == null) {
            continue;
          }
          

          to avoid the call to getSnapshotFiles.

          I'm not sure this is necessary. On one hand, it definitely saves 1 call stack. On the other hand, with the existence of all those loops and checks, I think the performance gain of saving 1 call stack would be trivial. And the nullity check of snapshottableDirs is already performed as a first step in getSnapshotFiles.

          Attached patch 005 with the above modifications. Updated summary below:

          • fsck -list-corruptfileblocks -includeSnapshots will also show full dir of snapshots
          • fsck -list-corruptfileblocks without -includeSnapshots will not show corrupt blocks that only have snapshot files
          • NameNode WebUI's way of showing corrupted files/blocks unchanged.
          • Added a sentence in NN WebUI to hint the admin to run fsck with -includeSnapshots, if there're snapshots present in the system.
          • Some refactoring to reuse existing code in new methods getSnapshottableDirs and ListCorruptFileBlocksWithSnapshot
          • The reasoning of keep minimal change to NN WebUI and fsck without -includeSnapshots is that getting all possible snapshots may be slow.
          Show
          xiaochen Xiao Chen added a comment - Thanks a lot for the review Yongjun Zhang ! 1. The description is not quite accurate per our discussion, suggest to modify. Especially the patch actually does change (and fix) the behavior when without -includeSnapshots. It was great to talk to you. I have updated the description. Modified patch summary in the end of this comment. 2. A possible optimization in FSDirSnapshotOp#getSnapshotFiles. It seems that the sf variable could be calculated in caller for once before the loop in the caller, and pass to this method. My apologies for the confusion, I added some comments in this method. But getting sf for each snapshottable dir is needed, since /d1 and /d2 have different snapshotlist. 3. final INodesInPath iip = fsd.getINodesInPath4Write(snap, false); maybe substituted with call to getINodesInPath Good catch! I updated the code to call getINode which invokes getINodesInPath . 4. The check if (!corruptFileBlocks.isEmpty()) in listCorruptFileBlocksWithSnapshot is not needed Good call. Fixed. 5. Add comment in listCorruptFileBlocks() before the call namenode.getNamesystem().listCorruptFileBlocksWithSnapshot, to indicate that snapshottableDirs is only relevant when -includeSnapshots is specified. Added a link to FSNamesystem#listCorruptFileBlocksWithSnapshot which explains that parameter in javadoc. 6. In listCorruptFileBlocksWithSnapshot, we can add if (snapshottableDirs == null ) { continue ; } to avoid the call to getSnapshotFiles. I'm not sure this is necessary. On one hand, it definitely saves 1 call stack. On the other hand, with the existence of all those loops and checks, I think the performance gain of saving 1 call stack would be trivial. And the nullity check of snapshottableDirs is already performed as a first step in getSnapshotFiles . Attached patch 005 with the above modifications. Updated summary below: fsck -list-corruptfileblocks -includeSnapshots will also show full dir of snapshots fsck -list-corruptfileblocks without -includeSnapshots will not show corrupt blocks that only have snapshot files NameNode WebUI's way of showing corrupted files/blocks unchanged. Added a sentence in NN WebUI to hint the admin to run fsck with -includeSnapshots, if there're snapshots present in the system. Some refactoring to reuse existing code in new methods getSnapshottableDirs and ListCorruptFileBlocksWithSnapshot The reasoning of keep minimal change to NN WebUI and fsck without -includeSnapshots is that getting all possible snapshots may be slow.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Hi Xiao Chen,

          Thanks for the new rev. Some further comments:

          1.

          My apologies for the confusion, I added some comments in this method. But getting sf for each snapshottable dir is needed, since /d1 and /d2 have different snapshotlist.

          I meant to precalcuate a list of "sf"s, and pass the list to FSDirSnapshotOp#getSnapshotFiles so to avoid doing it once for each file. Sorry for not having been clear.

          2.

          Added a link to FSNamesystem#listCorruptFileBlocksWithSnapshot which explains that parameter in javadoc.

          I think it's ok to not have this link since it looks a bit redundant. So consider I took back my earlier comment since you have javadoc in FSNamesystem#listCorruptFileBlocksWithSnapshot. sorry about that.

          3.

          Good catch! I updated the code to call getINode which invokes getINodesInPath.

          Good change to use getINode.

          4. In {{testFsckListCorruptSnapshotFiles() }}
          Suggest to introduce two variables, numFiles, numSnapshotFiles, initially numFile = 3 and numSnapshotFiles = 0, then when you create /corruptData/file, increment numFiles by 1, when you create snapshot, set numSnapshotFiles to 4. when deleting the file, decrement numFile by 1. When gets to assertion part, the first one check against (numFiles + numSnapshotFiles) (in place of the "7"), the second one check against numFiles.

          Thanks.

          Show
          yzhangal Yongjun Zhang added a comment - Hi Xiao Chen , Thanks for the new rev. Some further comments: 1. My apologies for the confusion, I added some comments in this method. But getting sf for each snapshottable dir is needed, since /d1 and /d2 have different snapshotlist. I meant to precalcuate a list of "sf"s, and pass the list to FSDirSnapshotOp#getSnapshotFiles so to avoid doing it once for each file. Sorry for not having been clear. 2. Added a link to FSNamesystem#listCorruptFileBlocksWithSnapshot which explains that parameter in javadoc. I think it's ok to not have this link since it looks a bit redundant. So consider I took back my earlier comment since you have javadoc in FSNamesystem#listCorruptFileBlocksWithSnapshot. sorry about that. 3. Good catch! I updated the code to call getINode which invokes getINodesInPath. Good change to use getINode . 4. In {{testFsckListCorruptSnapshotFiles() }} Suggest to introduce two variables, numFiles, numSnapshotFiles, initially numFile = 3 and numSnapshotFiles = 0, then when you create /corruptData/file, increment numFiles by 1, when you create snapshot, set numSnapshotFiles to 4. when deleting the file, decrement numFile by 1. When gets to assertion part, the first one check against (numFiles + numSnapshotFiles) (in place of the "7"), the second one check against numFiles. Thanks.
          Hide
          xiaochen Xiao Chen added a comment -

          Hi Yongjun,
          Thanks again for the quick review! Your comments makes sense, I attached patch 6 to address them. Sorry I missed the point of #1 earlier.

          Show
          xiaochen Xiao Chen added a comment - Hi Yongjun, Thanks again for the quick review! Your comments makes sense, I attached patch 6 to address them. Sorry I missed the point of #1 earlier.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 16m 35s Findbugs (version ) appears to be broken on trunk.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 8m 17s There were no new javac warning messages.
          +1 javadoc 10m 46s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 32s There were no new checkstyle issues.
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 38s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          -1 findbugs 2m 33s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 native 3m 22s Pre-build of native portion
          -1 hdfs tests 51m 37s Tests failed in hadoop-hdfs.
              96m 22s  



          Reason Tests
          FindBugs module:hadoop-hdfs
          Failed unit tests hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768348/HDFS-9231.005.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 600ad7b
          Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13161/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13161/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13161/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13161/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 16m 35s Findbugs (version ) appears to be broken on trunk. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 8m 17s There were no new javac warning messages. +1 javadoc 10m 46s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 32s There were no new checkstyle issues. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 38s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. -1 findbugs 2m 33s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 22s Pre-build of native portion -1 hdfs tests 51m 37s Tests failed in hadoop-hdfs.     96m 22s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768348/HDFS-9231.005.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 600ad7b Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13161/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13161/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13161/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13161/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Per the offline talk, we can make FSDirSnapshotOp#getSnapshotFiles more elegant by adding an interface on Snapshot to get the full snapshot root path, hence eliminating the need of pass in snapshottableDir. Attached patch 07 to do this. Thanks Yongjun.

          Show
          xiaochen Xiao Chen added a comment - Per the offline talk, we can make FSDirSnapshotOp#getSnapshotFiles more elegant by adding an interface on Snapshot to get the full snapshot root path, hence eliminating the need of pass in snapshottableDir. Attached patch 07 to do this. Thanks Yongjun.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 23m 51s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 10m 34s There were no new javac warning messages.
          +1 javadoc 13m 58s There were no new javadoc warning messages.
          +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 58s The applied patch generated 5 new checkstyle issues (total was 427, now 430).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 2m 1s mvn install still works.
          +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse.
          -1 findbugs 3m 30s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 native 4m 19s Pre-build of native portion
          -1 hdfs tests 63m 51s Tests failed in hadoop-hdfs.
              125m 24s  



          Reason Tests
          FindBugs module:hadoop-hdfs
          Failed unit tests hadoop.hdfs.TestDFSUpgradeFromImage



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768427/HDFS-9231.007.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 15eb84b
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13172/testReport/
          Java 1.7.0_55
          uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13172/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 23m 51s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 10m 34s There were no new javac warning messages. +1 javadoc 13m 58s There were no new javadoc warning messages. +1 release audit 0m 31s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 58s The applied patch generated 5 new checkstyle issues (total was 427, now 430). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 2m 1s mvn install still works. +1 eclipse:eclipse 0m 46s The patch built with eclipse:eclipse. -1 findbugs 3m 30s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 4m 19s Pre-build of native portion -1 hdfs tests 63m 51s Tests failed in hadoop-hdfs.     125m 24s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.TestDFSUpgradeFromImage Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768427/HDFS-9231.007.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 15eb84b Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13172/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13172/testReport/ Java 1.7.0_55 uname Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13172/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Patch 08 addresses checkstyle errors.
          The test failure is unrelate and passed locally. The findbugs warning is not visible from the link...

          Show
          xiaochen Xiao Chen added a comment - Patch 08 addresses checkstyle errors. The test failure is unrelate and passed locally. The findbugs warning is not visible from the link...
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 22m 1s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 9m 25s There were no new javac warning messages.
          +1 javadoc 11m 38s There were no new javadoc warning messages.
          +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 30s The applied patch generated 3 new checkstyle issues (total was 427, now 428).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 29s mvn install still works.
          +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse.
          -1 findbugs 2m 37s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 native 3m 17s Pre-build of native portion
          -1 hdfs tests 51m 11s Tests failed in hadoop-hdfs.
              104m 12s  



          Reason Tests
          FindBugs module:hadoop-hdfs
          Failed unit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.TestLeaseRecovery2



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768546/HDFS-9231.008.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 446212a
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13182/testReport/
          Java 1.7.0_55
          uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13182/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 22m 1s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 25s There were no new javac warning messages. +1 javadoc 11m 38s There were no new javadoc warning messages. +1 release audit 0m 25s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 30s The applied patch generated 3 new checkstyle issues (total was 427, now 428). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 29s mvn install still works. +1 eclipse:eclipse 0m 35s The patch built with eclipse:eclipse. -1 findbugs 2m 37s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 17s Pre-build of native portion -1 hdfs tests 51m 11s Tests failed in hadoop-hdfs.     104m 12s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768546/HDFS-9231.008.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 446212a Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13182/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13182/testReport/ Java 1.7.0_55 uname Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13182/console This message was automatically generated.
          Hide
          xiaochen Xiao Chen added a comment -

          Patch 009 fixes the check style warning of comments ending in '.'.
          Test failures, findbugs warning and other check style errors are unrelated.

          Show
          xiaochen Xiao Chen added a comment - Patch 009 fixes the check style warning of comments ending in '.'. Test failures, findbugs warning and other check style errors are unrelated.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Thanks Xiao for the new revs. +1 on 009 pending jenkins.

          Show
          yzhangal Yongjun Zhang added a comment - Thanks Xiao for the new revs. +1 on 009 pending jenkins.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          -1 pre-patch 23m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 9m 9s There were no new javac warning messages.
          +1 javadoc 11m 19s There were no new javadoc warning messages.
          +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 35s The applied patch generated 2 new checkstyle issues (total was 427, now 427).
          +1 whitespace 0m 1s The patch has no lines that end in whitespace.
          +1 install 1m 39s mvn install still works.
          +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse.
          -1 findbugs 2m 48s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings.
          +1 native 3m 33s Pre-build of native portion
          -1 hdfs tests 65m 46s Tests failed in hadoop-hdfs.
              120m 14s  



          Reason Tests
          FindBugs module:hadoop-hdfs
          Failed unit tests hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.namenode.ha.TestHASafeMode



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12768830/HDFS-9231.009.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 56e4f62
          Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13205/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13205/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment -1 pre-patch 23m 17s Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 9m 9s There were no new javac warning messages. +1 javadoc 11m 19s There were no new javadoc warning messages. +1 release audit 0m 26s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 35s The applied patch generated 2 new checkstyle issues (total was 427, now 427). +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 37s The patch built with eclipse:eclipse. -1 findbugs 2m 48s The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. +1 native 3m 33s Pre-build of native portion -1 hdfs tests 65m 46s Tests failed in hadoop-hdfs.     120m 14s   Reason Tests FindBugs module:hadoop-hdfs Failed unit tests hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.namenode.ha.TestHASafeMode Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12768830/HDFS-9231.009.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 56e4f62 Pre-patch Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt Findbugs warnings https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/13205/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13205/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13205/console This message was automatically generated.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Committed to trunk and branch-2. Thanks Xiao for the contribution.

          Show
          yzhangal Yongjun Zhang added a comment - Committed to trunk and branch-2. Thanks Xiao for the contribution.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8717 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8717/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8717 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8717/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2536 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2536/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2536 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2536/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #606 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/606/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #606 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/606/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1329 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1329/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1329 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1329/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #593 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/593/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #593 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/593/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2483 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2483/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2483 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2483/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #545 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/545/)
          HDFS-9231. fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #545 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/545/ ) HDFS-9231 . fsck doesn't list correct file path when Bad Replicas/Blocks (yzhang: rev 97913f430cbe3f82ac866ae6ab8f42754102f6c0) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NamenodeFsck.java hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.html hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirSnapshotOp.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeMXBean.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/Snapshot.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Yongjun for the reviews and commit!

          Show
          xiaochen Xiao Chen added a comment - Thanks Yongjun for the reviews and commit!
          Hide
          liuml07 Mingliang Liu added a comment -

          I found a findbugs warning in FSDirSnapshotOp probably brought by this patch, tracked by HDFS-9377. Please close that jira if it's wrong, or review that otherwise. Thanks.

          Show
          liuml07 Mingliang Liu added a comment - I found a findbugs warning in FSDirSnapshotOp probably brought by this patch, tracked by HDFS-9377 . Please close that jira if it's wrong, or review that otherwise. Thanks.
          Hide
          yzhangal Yongjun Zhang added a comment -

          Many thanks Mingliang Liu, I reviewed and commented in HDFS-9377.

          Show
          yzhangal Yongjun Zhang added a comment - Many thanks Mingliang Liu , I reviewed and commented in HDFS-9377 .
          Hide
          xiaochen Xiao Chen added a comment -

          Thanks Mingliang Liu and Yongjun Zhang, and sorry for the findbugs miss.

          Show
          xiaochen Xiao Chen added a comment - Thanks Mingliang Liu and Yongjun Zhang , and sorry for the findbugs miss.
          Hide
          liuml07 Mingliang Liu added a comment -

          Thanks for your quick reply, Yongjun Zhang and Xiao Chen.

          Actually the Jenkins test-patch report is not showing this findbugs warning and it's understood to miss it. I also hope the HADOOP-12517 be resolved soon, if there is any fundamental issue.

          Show
          liuml07 Mingliang Liu added a comment - Thanks for your quick reply, Yongjun Zhang and Xiao Chen . Actually the Jenkins test-patch report is not showing this findbugs warning and it's understood to miss it. I also hope the HADOOP-12517 be resolved soon, if there is any fundamental issue.

            People

            • Assignee:
              xiaochen Xiao Chen
              Reporter:
              xiaochen Xiao Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development