Details
-
Type:
Improvement
-
Status: Resolved
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 2.7.1
-
Fix Version/s: 2.8.0, 3.0.0-alpha1
-
Component/s: test
-
Labels:None
-
Target Version/s:
-
Hadoop Flags:Reviewed
Description
Currently, HDFS does block corruption tests by directly accessing the files stored on the storage directories, which assumes FsDatasetImpl is the dataset implementation. However, with works like OZone (HDFS-7240) and HDFS-8679, there will be different FsDataset implementations.
So we need a general way to run whitebox tests like corrupting blocks and crc files.
Issue Links
- is required by
-
HDFS-9282
Make data directory count and storage raw capacity related tests FsDataset-agnostic
-
- Resolved
-
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
 -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 10m 23s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 11 new or modified test files. |
| +1 | javac | 9m 58s | There were no new javac warning messages. |
| -1 | release audit | 0m 16s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 47s | There were no new checkstyle issues. |
| -1 | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. |
| +1 | install | 1m 43s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 37s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 51s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 36s | Pre-build of native portion |
| -1 | hdfs tests | 61m 25s | Tests failed in hadoop-hdfs. |
| 90m 43s |
| Reason | Tests |
|---|---|
| Timed out tests | org.apache.hadoop.fs.TestGlobPaths |
| org.apache.hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12764687/HDFS-9188.000.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / fd026f5 |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12768/artifact/patchprocess/patchReleaseAuditProblems.txt |
| whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/12768/artifact/patchprocess/whitespace.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12768/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12768/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12768/console |
This message was automatically generated.
Address release audit and whitespace warnings.
The test failures are not relevant.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 8m 4s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 11 new or modified test files. |
| +1 | javac | 7m 59s | There were no new javac warning messages. |
| -1 | release audit | 0m 13s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 26s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 1s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 28s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 30s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 17s | Pre-build of native portion |
| -1 | hdfs tests | 191m 35s | Tests failed in hadoop-hdfs. |
| 215m 11s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
| hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles | |
| hadoop.hdfs.TestFileCreation | |
| hadoop.hdfs.web.TestWebHDFSOAuth2 | |
| hadoop.hdfs.server.namenode.TestProcessCorruptBlocks | |
| hadoop.hdfs.util.TestByteArrayManager |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12764834/HDFS-9188.001.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 1037ee5 |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12776/artifact/patchprocess/patchReleaseAuditProblems.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12776/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12776/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf900.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12776/console |
This message was automatically generated.
thanks, Lei (Eddy) Xu.
ReplicaToCorrupt: it seems like this should be named something like MaterializedReplica. Its distinguishing factor is that it is the concrete representation of some replica in the FSDataset.
92 /** 93 * Corrupt the block file by deleting it. 94 * @return true if the deletion is completed. 95 */ 96 boolean deleteData();
This should be able to throw an IOE. Same with deleteMeta.
Thanks a lot for the reviews, Colin P. McCabe. I updated the patch to address your comments.
Please take a look.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 7m 54s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 11 new or modified test files. |
| +1 | javac | 8m 11s | There were no new javac warning messages. |
| -1 | release audit | 0m 14s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 27s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 0s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 31s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 33s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 16s | Pre-build of native portion |
| -1 | hdfs tests | 189m 29s | Tests failed in hadoop-hdfs. |
| 213m 12s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.server.namenode.TestProcessCorruptBlocks |
| hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12764889/HDFS-9188.002.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / fdf02d1 |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12782/artifact/patchprocess/patchReleaseAuditProblems.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12782/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12782/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12782/console |
This message was automatically generated.
Fix TestProcessCorruptBlocks.
Could not reproduce TestScrLazyPersistFiles.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 9m 1s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 12 new or modified test files. |
| +1 | javac | 8m 43s | There were no new javac warning messages. |
| -1 | release audit | 0m 13s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 32s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 1s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 34s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 37s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 39s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 17s | Pre-build of native portion |
| -1 | hdfs tests | 208m 33s | Tests failed in hadoop-hdfs. |
| 234m 15s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.server.namenode.ha.TestDNFencing |
| hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles | |
| hadoop.hdfs.web.TestWebHdfsTimeouts | |
| hadoop.hdfs.security.TestDelegationTokenForProxyUser | |
| hadoop.hdfs.server.namenode.TestProcessCorruptBlocks | |
| hadoop.hdfs.web.TestWebHdfsContentLength | |
| hadoop.hdfs.server.namenode.TestFSNamesystem |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12765096/HDFS-9188.003.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 30ac69c |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12803/artifact/patchprocess/patchReleaseAuditProblems.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12803/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12803/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12803/console |
This message was automatically generated.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 8m 7s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 12 new or modified test files. |
| +1 | javac | 7m 56s | There were no new javac warning messages. |
| -1 | release audit | 0m 15s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 1s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 1s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 29s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 33s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 4s | Pre-build of native portion |
| -1 | hdfs tests | 191m 4s | Tests failed in hadoop-hdfs. |
| 214m 7s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.server.blockmanagement.TestBlockManager |
| hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles | |
| hadoop.hdfs.server.namenode.TestProcessCorruptBlocks |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12765096/HDFS-9188.003.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / 99e5204 |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12840/artifact/patchprocess/patchReleaseAuditProblems.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12840/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12840/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12840/console |
This message was automatically generated.
Thanks, Lei (Eddy) Xu.
38 abstract class Factory<D extends FsDatasetTestUtils> { 39 /** 40 * By default returns the Flood version of the factory. 41 * 42 * @return The configured Factory. 43 */
This comment seems out of place.
26 public class FsDatasetTestUtilsFactory 27 extends FsDatasetTestUtils.Factory<FsDatasetTestUtils> { 28 @Override 29 public FsDatasetTestUtils newInstance(DataNode datanode) { 30 return new FsDatasetImplTestUtils(datanode); 31 }
The factory name should be FsDatasetImplTestUtilsFactory, since it's returning an instance of FsDatasetImplTestUtils. Also, this should be final.
60 private static boolean corrupt(File file) throws IOException { 61 if (file == null || !file.exists()) { 62 return false; 63 } 64 LOG.debug("Corrupting file: " + file); 65 StringBuilder content = new StringBuilder(DFSTestUtil.readFile(file)); 66 char c = content.charAt(0); 67 content.setCharAt(0, ++c); 68 try (PrintWriter out = new PrintWriter(file)) { 69 out.print(content); 70 out.flush(); 71 } 72 return true;
This is assuming that the content is a string. It would be better to just manipulate the file as a binary array. This also doesn't specify the string encoding (unfortunately, we can't take UTF-8 for granted here).
88 /** Corrupt a block / crc file by deleting it. */ 89 private static boolean delete(File file) { 90 if (file == null || !file.exists()) { 91 return false; 92 } 93 return file.delete(); 94 }
Please use Files#delete, which throws an exception when it fails. And don't return a boolean. See http://docs.oracle.com/javase/7/docs/api/java/nio/file/Files.html#delete(java.nio.file.Path)
101 @Override 102 public boolean corruptData() throws IOException { 103 LOG.info("Corrupting block file: " + blockFile); 104 return corrupt(blockFile); 105 }
Why does this return a boolean? If it fails it should throw an exception with the reason why it failed. Same question with truncate.
If we need a MaterializedReplica#exists method then let's add one rather than making everything return false if the replica doesn't exist.
Also, shouldn't we fail to create the materialized replica if the underlying files don't exist? There should be some standard exception that is thrown in this case like ReplicaNotFoundException.
Thanks a lot for the detailed reviews, Colin P. McCabe.
I updated the patch to address your comments. Could you give another review? Thanks!
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| 0 | pre-patch | 8m 16s | Pre-patch trunk compilation is healthy. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 15 new or modified test files. |
| +1 | javac | 8m 9s | There were no new javac warning messages. |
| -1 | release audit | 0m 16s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 24s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 2s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 29s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 29s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 3s | Pre-build of native portion |
| -1 | hdfs tests | 189m 43s | Tests failed in hadoop-hdfs. |
| 213m 28s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles |
| hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints | |
| hadoop.hdfs.TestReplication | |
| hadoop.hdfs.TestReadStripedFileWithDecoding | |
| hadoop.hdfs.server.mover.TestMover | |
| hadoop.hdfs.TestDatanodeDeath | |
| Timed out tests | org.apache.hadoop.hdfs.TestDFSStripedOutputStreamWithFailure000 |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12765882/HDFS-9188.004.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / c32614f |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12896/artifact/patchprocess/patchReleaseAuditProblems.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12896/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12896/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf901.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12896/console |
This message was automatically generated.
Some of the test failures are relevant. Working on them.
Updated the patch to address test failures.
Thanks, Lei (Eddy) Xu.
1533 for(MaterializedReplica replica : replicas) {
1534 replica.corruptData(sb.toString().getBytes());
1535 }
You need to set UTF8 as the encoding.
+1 once that's resolved and pending jenkins
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| -1 | pre-patch | 7m 53s | Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 15 new or modified test files. |
| +1 | javac | 8m 5s | There were no new javac warning messages. |
| +1 | release audit | 0m 21s | The applied patch does not increase the total number of release audit warnings. |
| +1 | checkstyle | 1m 30s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 1s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 26s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 26s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 4s | Pre-build of native portion |
| -1 | hdfs tests | 49m 17s | Tests failed in hadoop-hdfs. |
| 72m 39s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.TestLeaseRecovery2 |
| hadoop.fs.TestGlobPaths |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12766622/HDFS-9188.005.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / dfa7848 |
| Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12987/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12987/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12987/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12987/console |
This message was automatically generated.
Thanks a lot for the reviews, Colin P. McCabe.
You need to set UTF8 as the encoding.
Fixed.
The test failures are not relevant.
| -1 overall |
| Vote | Subsystem | Runtime | Comment |
|---|---|---|---|
| -1 | pre-patch | 8m 3s | Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. |
| +1 | @author | 0m 0s | The patch does not contain any @author tags. |
| +1 | tests included | 0m 0s | The patch appears to include 14 new or modified test files. |
| +1 | javac | 8m 4s | There were no new javac warning messages. |
| -1 | release audit | 0m 16s | The applied patch generated 1 release audit warnings. |
| +1 | checkstyle | 1m 25s | There were no new checkstyle issues. |
| +1 | whitespace | 0m 1s | The patch has no lines that end in whitespace. |
| +1 | install | 1m 30s | mvn install still works. |
| +1 | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. |
| +1 | findbugs | 2m 27s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. |
| +1 | native | 1m 2s | Pre-build of native portion |
| -1 | hdfs tests | 53m 0s | Tests failed in hadoop-hdfs. |
| 76m 26s |
| Reason | Tests |
|---|---|
| Failed unit tests | hadoop.hdfs.TestRecoverStripedFile |
| hadoop.hdfs.TestLeaseRecovery2 | |
| hadoop.fs.TestGlobPaths | |
| hadoop.hdfs.server.namenode.TestFileTruncate |
| Subsystem | Report/Notes |
|---|---|
| Patch URL | http://issues.apache.org/jira/secure/attachment/12766665/HDFS-9188.006.patch |
| Optional Tests | javac unit findbugs checkstyle |
| git revision | trunk / be7a0ad |
| Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12995/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html |
| Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12995/artifact/patchprocess/patchReleaseAuditProblems.txt |
| hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12995/artifact/patchprocess/testrun_hadoop-hdfs.txt |
| Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12995/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12995/console |
This message was automatically generated.
Thanks a lot for the reviews, Colin P. McCabe.
Committed to trunk and branch-2.
FAILURE: Integrated in Hadoop-trunk-Commit #8639 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8639/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
SUCCESS: Integrated in Hadoop-Yarn-trunk-Java8 #546 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/546/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk #2482 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2482/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
SUCCESS: Integrated in Hadoop-Yarn-trunk #1269 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1269/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #533 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/533/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
FAILURE: Integrated in Hadoop-Hdfs-trunk #2436 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2436/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #499 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/499/)
HDFS-9188. Make block corruption related tests FsDataset-agnostic. (lei) (lei: rev c80b3a804f5222f95a266f84424af9cb9c229483)
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestMissingBlocksAlert.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestFileAppend3.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestProcessCorruptBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestOverReplicatedBlocks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetImplTestUtilsFactory.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/FsDatasetTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/FsDatasetImplTestUtils.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestBlocksWithNotEnoughRacks.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestClientReportBadBlock.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestRBWBlockInvalidation.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/fsdataset/impl/TestScrLazyPersistFiles.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSShell.java
- hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/DataNodeTestUtils.java
The patch: