[HADOOP-18989] Use thread pool to improve the speed of creating control files in TestDFSIO

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.6
• Fix Version/s: 3.4.0
• Component/s: benchmarks, common
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: farmmamba
• Reporter: farmmamba
• Created: 24/Nov/23 06:33
• Updated: 08/Dec/23 12:52
• Resolved: 08/Dec/23 09:18

Description

When we use TestDFSIO tool to test the throughouts of HDFS clusters, we found it is so slow in the creating controll files stage. 

After refering to the source code, we found that method createControlFile try to create control files serially. It can be improved by using thread pool.

After optimizing, the TestDFSIO tool runs quicker than before.

 

Issue Links

• links to: GitHub Pull Request #6294