[HDFS-9282] Make data directory count and storage raw capacity related tests FsDataset-agnostic

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.7.1
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: test
• Assignee: Tony Wu
• Reporter: Tony Wu
• Created: 22/Oct/15 00:11
• Updated: 12/May/16 18:12
• Resolved: 05/Nov/15 18:48

Description

DFSMiniCluster and several tests have hard coded assumption of the underlying storage having 2 data directories (volumes). As HDFS-9188 pointed out, with new FsDataset implementations, these hard coded assumption about number of data directories and raw capacities of storage may change as well.

We need to extend FsDatasetTestUtils to provide:

• Number of data directories of underlying storage per DataNode
• Raw storage capacity of underlying storage per DataNode.
• Have MiniDFSCluster automatically pick up the correct values.

Issue Links

• requires: HDFS-9188 Make block corruption related tests FsDataset-agnostic.