[HDFS-10891] Über-jira: Test Improvement by adding missing test cases for existing code

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: test
• Target Version/s: 2.8.0
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 23/Sep/16 00:33
• Updated: 29/Dec/16 00:30
• Resolved: 29/Dec/16 00:30

Description

We have covered some test cases in our nightly run and we'd like to backport those test cases to the community code branch. This work includes the efforts but not limited to:

1. Add more meaningful test cases for existing code so it's hard to break it as stability and compatibility should always be guarded if possible
2. Refactor existing tests that share common code snippet (e.g. helper methods) and/or logic (e.g. set up a MiniDFSCluster). One approach that is not popular (which it should) is @Parameterized tests
3. Reduce unnecessary overhead in unit tests (e.g. long interval sleep, build MiniDFSCluster multiple times instead of reusing the same one). Some of the cases were addressed by HDFS-10666.

This is not a long-term work to cover all future improvement in unit tests. We'll be happy to resolve this after our internal test cases (will file separate JIRAs for that) are mostly addressed/considered.

Issue Links

• contains: HDFS-10986 DFSAdmin should log detailed error message if any
• contains: HADOOP-13686 Adding additional unit test for Trash (I)
• relates to: HDFS-10666 Über-jira: Unit tests should not use fixed sleep interval to wait for conditions

Sub-Tasks

• Add unit tests for HDFS command 'dfs -tail' and 'dfs -stat': Resolved
• Refactor TestDFSShell by setting up MiniDFSCluser once for all commands test: Resolved
• Add unit tests for Trash with HDFS encryption zones: Resolved
• Adding additional unit tests for Trash (II): Resolved
• TestDFSShell.testStat fails intermittently: Resolved
• Add unit tests to verify ACLs in safemode: Closed
• Add more unit tests for ACLs: Open
• Add unit test for HDFS command 'dfsadmin -printTopology': Resolved
• Add unit test for HDFS command 'dfsadmin -getDatanodeInfo': Resolved
• Add unit tests for HDFS command 'dfsadmin -fetchImage' in HA: Resolved
• Change unit test for testing parsing "-source" parameter in Balancer CLI: Resolved
• Add unit tests for HDFS command 'dfsadmin -set/clrSpaceQuota': Resolved
• Add unit tests for HDFS command 'dfsadmin -set/clrQuota': Closed
• TestDataNodeVolumeFailure#testVolumeFailure is flaky (though passing): Resolved
• Add additional unit test for DataNode startup behavior when volumes fail: Resolved
• Add space quota tests for heterogenous storages: Resolved
• Add unit test for extended Acls: Resolved
• Add unit test for DFSAdmin -report command: Resolved
• Accelerate TestQuotaByStorageType using shared MiniDFSCluster: Closed
• Add unit test for NameNode failing to start when name dir is unwritable: Resolved
• Accelerate TestDiskBalancerCommand using static shared MiniDFSCluster: Closed
• TestDFSAdmin#testReportCommand fails due to timed out: Resolved