[HDFS-1852] Umbrella task: Clean up HDFS unit test recurring failures

• Type: Test
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.22.0
• Component/s: test
• Assignee: Unassigned
• Reporter: Matthew Foley
• Created: 21/Apr/11 05:35
• Updated: 16/Aug/11 00:34

Description

Recurring failures and false positives undermine CI by encouraging developers to ignore unit test failures. Let's clean these up!

Some are intermittent due to timing-sensitive conditions. The unit tests for background thread activities (such as block replication and corrupt replica detection) often use "wait while" or "wait until" loops to detect results. The quality and robustness of these loops vary widely, and common usages should be moved to DFSTestUtil.

Issue Links

• is blocked by: HDFS-2232 TestHDFSCLI fails on 0.22 branch

Sub-Tasks

• make failure message more useful in DFSTestUtil.waitReplication(): Closed
• TestDatanodeBlockScanner waits forever, errs without giving information: Closed
• Recurring failures in TestDatanodeBlockScanner since HDFS-1052: Resolved
• Recurring failures in TestBackupNode since HDFS-1052: Closed
• Improve TestDFSStorageStateRecovery: Closed
• Recurring failures in TestFileAppend4, TestLargeBlock, TestWriteConfigurationToDFS for > 12 days: Resolved
• TestFileConcurrentReader test case is still timing out / failing: Resolved
• Recurring failure in TestJMXGet.testNameNode since build 477 on May 11: Closed
• Intermittent recurring failure in TestFiDataTransferProtocol2.pipeline_Fi_29: Closed
• TestBlocksWithNotEnoughRacks intermittently fails assert: Closed
• TestHDFSTrash failing on trunk and 22: Resolved