[HDFS-9379] Make NNThroughputBenchmark$BlockReportStats support more than 10 datanodes

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: test
• Hadoop Flags: Reviewed
• Assignee: Mingliang Liu
• Reporter: Mingliang Liu
• Created: 05/Nov/15 03:35
• Updated: 30/Aug/16 01:23
• Resolved: 07/Nov/15 03:00

Description

Currently, the NNThroughputBenchmark test BlockReportStats relies on sorted datanodes array in the lexicographical order of datanode's xferAddr.

• There is an assertion of datanode's xferAddr lexicographical order when filling the datanodes, see the code.
• When searching the datanode by DatanodeInfo, it uses binary search against the datanodes array, see the code

In DatanodeID, the xferAddr is defined as host:port. In NNThroughputBenchmark, the port is simply the index of the tiny datanode plus one.

The problem here is that, when there are more than 9 tiny datanodes (numThreads), the lexicographical order of datanode's xferAddr will be invalid as the string value of datanode index is not in lexicographical order any more. For example,

...
192.168.54.40:8
192.168.54.40:9
192.168.54.40:10
192.168.54.40:11
...

192.168.54.40:9 is greater than 192.168.54.40:10. The assertion will fail and the binary search won't work.

The simple fix is to calculate the datanode index by port directly, instead of using binary search.

Issue Links

• is cloned by: HDFS-9436 Make NNThroughputBenchmark$BlockReportStats run with 10 datanodes by default