[HDFS-8381] Reduce time taken for complete HDFS unit test run

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: test
• Assignee: Unassigned
• Reporter: Arpit Agarwal
• Created: 12/May/15 23:44
• Updated: 14/May/15 20:48

Description

HDFS unit tests take a long time to run. Our unit tests are more like system/integration tests since we spin up a MiniDFSCluster for individual test cases. A number of tests have sleeps which further adds to the run time.

A better option is to use more fine-grained unit tests specific to individual classes. I did not find any existing Jiras for this so filing one to track this work.

Sub-Tasks

• Eliminate retries in TestFileCreation#testOverwriteOpenForWrite: Resolved
• Remove unnecessary throttling in TestDatanodeDeath: Resolved
• Consider skipping stress tests for precommit builds: Open
• Reduce runtime for append test: Resolved