[HDFS-15058] TestFsck.testFsckListCorruptFilesBlocks and TestFsck.testFsckListCorruptSnapshotFiles fail some times

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Assignee: Unassigned
• Reporter: liusheng
• Created: 13/Dec/19 07:12
• Updated: 16/Dec/19 01:09
• Resolved: 13/Dec/19 14:18

Description

when I am try to run the tests of HDFS, the TestFsck.testFsckListCorruptFilesBlocks and TestFsck.testFsckListCorruptSnapshotFiles tests are easy to fail, see: 

06:26:38 [ERROR] Failures: 
06:26:38 [ERROR]   TestFsck.testFsckListCorruptFilesBlocks:1167
06:26:38 [ERROR]   TestFsck.testFsckListCorruptSnapshotFiles:2167
06:26:38 [INFO]
06:26:38 [ERROR] Tests run: 33, Failures: 2, Errors: 0, Skipped: 0

Both of these two tests failures are mainly because the tests will check the number of corrupt files after sleep 1000 ms and the number is not equal to expected. see:

blk_1073741825  /corruptData/8117051706407353421
blk_1073741825  /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2. bad fsck include snapshot out: The list of corrupt files under path '/corruptData' are:
blk_1073741825  /corruptData/8117051706407353421
blk_1073741825  /corruptData/.snapshot/mySnapShot/8117051706407353421
The filesystem under path '/corruptData' has 2 CORRUPT files2019-12-13 06:26:35,808 [Listener at localhost/44367] INFO  hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdown(2067)) - Shutting down the Mini HDFS Cluster
2019-12-13 06:26:35,808 [Listener at localhost/44367] INFO  hdfs.MiniDFSCluster (MiniDFSCluster.java:shutdownDataNode(2115)) - Shutting down DataNode 0

To fix these two tests, we need to enlarge the sleep time of 1000 ms, according to my testing, enlaging the time to 5000 ms can make the tests passed every times. 

Issue Links

• duplicates: HDFS-15038 TestFsck testFsckListCorruptSnapshotFiles is failing in trunk
• links to: GitHub Pull Request #1760