[HDFS-10666] Über-jira: Unit tests should not use fixed sleep interval to wait for conditions

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: test
• Target Version/s: 2.8.0
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 21/Jul/16 06:07
• Updated: 12/Nov/16 03:24
• Resolved: 12/Nov/16 03:24

Description

There have been dozens of intermittent failing unit tests because they depend on fixed-interval sleep to wait for conditions to reach before assertion. This umbrella jira is to replace these sleep statements with:

• GenericTestUtils.waitFor() to retry the conditions/assertions
• Trigger internal state change of code to test, e.g. for MiniDFSCluster we can trigger {BlockReports,HeartBeats,DeletionReports}
• Fails fast if specific exceptions are caught
• Coordinate the JUnit thread with activities of the internal threads
• ad-hoc fixes...

p.s. I don't know how closures in Java 8 comes into play but I'd like to see any effort.

Issue Links

• contains: HADOOP-10823 TestReloadingX509TrustManager is flaky
• contains: HADOOP-13375 o.a.h.security.TestGroupsCaching.testBackgroundRefreshCounters seems flaky
• is related to: HDFS-10891 Über-jira: Test Improvement by adding missing test cases for existing code

Sub-Tasks

• TestDataNodeMXBean#testDataNodeMXBeanBlockCount fails intermittently: Resolved
• TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently: Resolved
• o.a.h.hdfs.server.blockmanagement.TestBlockManagerSafeMode.testCheckSafeMode fails intermittently: Resolved
• o.a.h.hdfs.server.namenode.ha.TestPendingCorruptDnMessages fails intermittently: Resolved
• TestBlockManagerSafeMode#testCheckSafeMode fails intermittently: Resolved
• o.a.h.hdfs.TestLeaseRecovery2 fails intermittently: Resolved
• TestBlockManager#testBlockReportQueueing fails intermittently: Resolved
• o.a.h.ha.TestZKFailoverController should not use fixed time sleep before assertions: Resolved
• Refactor TestBalancerBandwidth: Resolved
• TestNameNodeMetadataConsistency#testGenerationStampInFuture Fails Intermittently: Resolved
• TestRBWBlockInvalidation#testRWRInvalidation fails intermittently: Resolved
• TestAddStripedBlockInFBR#testAddBlockInFullBlockReport fails frequently in trunk: Resolved